Back when I started this site around 2020 it was pretty much just me here. Most of the time the new user applications were from ultra lazy spammers peddling penis pills. I realized that all of you have been writing introductions but you don’t know much about me so here is somewhat of an introduction.
I’m just a regular guy in the Cleveland, Ohio area who works in tech. I have 3 cats. My hobbies are tech stuff, shooting/guns, gardening, lifting, and to a smaller degree talking about leftist politics. I run this instance as well as I can on my own. Sometimes the site will go down or be slow but you can rest assured that I’m most likely aware of it when that happens. I monitor this site pretty closely.
It definitely feels weird (in a good but anxiety inducing way) to see so many people using the server I set up. Before the influx it didn’t matter if the site was down for an hour or two while I worked out a deployment issue but that’s not the case anymore. I would like to bring more sys admins on board eventually so if you’re interested feel free to reach out.
Glad you’re all here!
Thanks for hosting the server. Scaling while being slammed is tough! I’ve noticed a few times the server is too loaded and try to come back later. With reddit down you’re essentially feeling ‘the slashdot effect’ (is that still a thing? lol). I’ll keep coming back and checking =)
I’ve done infra/ops for years, and been in your situation more than once. Take your time, you’ll get it all sorted out. John Allspaw wrote a good book about this subject, “The Art of Capacity Planning: Scaling Web Resources”. There is a new edition out that I haven’t read, but all of the lessons learned in that book are still applicable today. The biggest thing that helped me was learning haproxy and figuring out how to spread out the load, minimize response times, as well as queue time.
So to a lay person, are the speed/responsiveness issues just because so many people are using all the servers, or is it this instance, or the Lemmy software (or a bit of all 3)? I tried posting a simple link too [email protected] and it just spun the whole time.
I would guess having not seen any logs or anything that the server might need more resources or is hitting some limit that hasn’t come up before. Like max open files or some sort of ram limit. I haven’t looked into running a lemmy instance yet but usually you can mess with the config and squeeze a little more life out of stuff. But it could be that the server itself needs to be more powerful.
I’ve had a lot of problems getting posts to load that are on other large instances. The whole ecosystem is probably just slammed right now. Reddit dying made people go “well whats a thing like reddit I could use instead” and we’re all just showing up load testing the small instances people set up for a small group of friends.
I like to think all of our Midwest internet traffic is meeting at the server and saying “Ope, sorry, after you…No I insist, after you.”