It says 16gb of vram in the first line of the article. My 8gb kills me. Its a beast of a card, buy as soon as i go over the vram limit, it slows to a crawl.
Their top tier 7800 XTX had 24GB. Most AI models need at least 24 but preferably 32. Guess they don’t need to try when NVIDIA isn’t either, despite not being very expensive to do so.
Most AI models need at least 24 but preferably 32.
Where are you getting this information from? Most models that are less than 16B params will run just fine with less than 24 GB of VRAM. This github discussion thread for open-webui (a frontend for Ollama) has a decent reference for VRAM requirements.
I should have been more specific. The home models that actually compete with paid ones in both accuracy & speed. Please don’t be one of those to exaggerate & pretend it works just as good with much less. It simply doesn’t.
But will it have enough RAM
It says 16gb of vram in the first line of the article. My 8gb kills me. Its a beast of a card, buy as soon as i go over the vram limit, it slows to a crawl.
Their top tier 7800 XTX had 24GB. Most AI models need at least 24 but preferably 32. Guess they don’t need to try when NVIDIA isn’t either, despite not being very expensive to do so.
Where are you getting this information from? Most models that are less than 16B params will run just fine with less than 24 GB of VRAM. This github discussion thread for open-webui (a frontend for Ollama) has a decent reference for VRAM requirements.
I should have been more specific. The home models that actually compete with paid ones in both accuracy & speed. Please don’t be one of those to exaggerate & pretend it works just as good with much less. It simply doesn’t.
Maybe with CXL or infinityfabric that won’t matter as much.
Don’t know what they’re support looks like on the consumer side though