@[email protected] to [email protected]English • 6 months agoHow Gradient created an open LLM with a million-token context windowventurebeat.comexternal-linkmessage-square4fedilinkarrow-up126arrow-down13cross-posted to: [email protected]aicompanions
arrow-up123arrow-down1external-linkHow Gradient created an open LLM with a million-token context windowventurebeat.com@[email protected] to [email protected]English • 6 months agomessage-square4fedilinkcross-posted to: [email protected]aicompanions
minus-square@TechNerdWizard42link4•6 months agoI believe you’d need roughly 500GB of RAM to run it minimum at full context length. There is chatter that 125k context took and used 40GB I know I can load the 70B models into my laptop at lower bits but it consumes about 140GB of RAM.
I believe you’d need roughly 500GB of RAM to run it minimum at full context length. There is chatter that 125k context took and used 40GB
I know I can load the 70B models into my laptop at lower bits but it consumes about 140GB of RAM.