@Wilshire to TechnologyEnglish • 5 months agoThe first GPT-4-class AI model anyone can download has arrived: Llama 405Barstechnica.comexternal-linkmessage-square61arrow-up1215arrow-down117cross-posted to: [email protected]
arrow-up1198arrow-down1external-linkThe first GPT-4-class AI model anyone can download has arrived: Llama 405Barstechnica.com@Wilshire to TechnologyEnglish • 5 months agomessage-square61cross-posted to: [email protected]
minus-squaresunzulinkfedilink2•5 months agoDid anyone get 70b to run locally? If so what, what hardware specs?
minus-squaresunzulinkfedilink2•5 months agoI guess time to buy some ram after spending decade at 16gb
Did anyone get 70b to run locally?
If so what, what hardware specs?
Afaik you need about 40GB of vram for a 70b model.
Can’t you offload some of it to RAM?
Same requirements, but much slower.
I guess time to buy some ram after spending decade at 16gb