I successfully installed oobavooga on a PC, and by using only the CPU I can run a vicuna and a wizardlm model. I can’t run any llama, opt, gpt-j which looks required when trying to train the model through Lora. Do you have any suggestion?
I successfully installed oobavooga on a PC, and by using only the CPU I can run a vicuna and a wizardlm model. I can’t run any llama, opt, gpt-j which looks required when trying to train the model through Lora. Do you have any suggestion?
While it won’t help OP, there are GPUs that have plenty of VRAM but are lacking CUDA.
Apple sells some with 192GB of VRAM for example… and as a Mac user I’m pretty frustrated by all the dependencies on CUDA. The few tools that don’t require it run great on my Mac (Stable Diffusion for example, runs in 30 seconds on my laptop with typical settings).