Does that mean that their target model size is 7B? I can’t see anything larger fitting into 16GB without disrupting regular system operations. The 40 TOPS number is pretty close to the 39 TOPS that the Ryzen 8000 NPUs are supposed to have so It seems like they aren’t necessarily targeting systems with dedicated cards.
Does that mean that their target model size is 7B? I can’t see anything larger fitting into 16GB without disrupting regular system operations. The 40 TOPS number is pretty close to the 39 TOPS that the Ryzen 8000 NPUs are supposed to have so It seems like they aren’t necessarily targeting systems with dedicated cards.