A simple question to this community, what are you self-hosting? It’s probably fun to hear from each-other what services we are running.
Please mention at least the service (e.g. e-mail) and the software (e.g. postfix). Extra bonus points for also mentioning the OS and/or hardware (e.g. Linux Distribution, raspberry pi, etc) you are running on.
Stable Diffusion (Stability AI version), text-generation-webui (WizardLM), a text embedder service with Spacy, Bert and a bunch of sentence-transformer models, PiHole, Octoprint, Elasticsearch/Kibana for my IoT stuff, Jellyfin, Sonarr, FTB Minecraft (customized pack), a few personal apps I wrote myself (todo lists), SMB file shares, qBittorrent and Transmission (one dedicated to Sonarr)… Probably a ton of other stuff I’m forgetting.
Do you have a GPU in there with the Stable Diffusion? If not how’s it working? I’m debating moving to a machine I can’t guarantee my spare GPU will fit in.2
Without a GPU, it’s pretty horrible.
That’s what I was afraid of. Wad gifted a Dell VRTX blade chassis and trying to figure out how to shove my spare 2080 Super in there and get it power.
Yep, I’m using an RTX2070 for that right now. The LLMs are just executing on CPU.