A while ago, I had requested help with using LLMs to manage all my teaching notes. I have since installed Ollama and been playing with it to get a feel for the setup.
I was also suggested the use of RAG (Retrieval Augmented Generation ) and CA (cognitive architecture). However, I am unclear on good self hosted options for these two tasks. Could you please suggest a few?
For example, I tried ragflow.io and installed it on my system, but it seems I need to setup an account with a username and password to use it. It remains unclear if I can use the system offline like the base ollama model, and that information won’t be sent from my computer system.
I’m not 100% what you’re asking for, but I use text-generation-webui for all of my local generation needs.
https://github.com/oobabooga/text-generation-webui
Text-generation-webui is cool, but also kinda crufty. Honestly a lot of the stuff is holdovers from what’s now ancient history in LLM land, and it has (for me) major performance issues at longer context.
Anything better you know of? Most of my usage now with it is through its api
Uh, depends on your hardware and model, but probably TabbyAPI?
Neat! I’ll check it out!