As the AI market continues to balloon, experts are warning that its VC-driven rise is eerily similar to that of the dot com bubble.
As the AI market continues to balloon, experts are warning that its VC-driven rise is eerily similar to that of the dot com bubble.
Oh wow, that’s good to know. I always attributed visual graphics to be way more intensive. Wouldn’t think a text generative model to take up that much Vram
Edit: how many parameters did you test with?
Sorry, just seeing this now- I think with 24gb of vram, the most you can get is a 4bit quantized 30b model, and even then, I think you’d have to limit it to 2-3k of context. Here’s a chart for size comparisons: https://postimg.cc/4mxcM3kX
By comparison, with 24gb of vram, I only use half of that to create a batch of 8 768x576 photos. I also sub to mage.space, and I’m pretty sure they’re able to handle all of their volume on an A100 and A10G