I’m really enjoying lemmy. I think we’ve got some growing pains in UI/UX and we’re missing some key features (like community migration and actual redundancy). But how are we going to collectively pay for this? I saw an (unverified) post that Reddit received 400M dollars from ads last year. Lemmy isn’t going to be free. Can someone with actual server experience chime in with some back of the napkin math on how expensive it would be if everyone migrated from Reddit?
As paradoxical as it is, I think that these open source non-profit projects are a lot more efficient than profit-driven, debt-fueled corporations.
First of all, the main contributors to a FOSS project do it for passion and do not take a salary.
Secondly, they don’t have the infinite growth mindset that pushes enterpreneurs to to spend as much as possible for maximum growth, all financed by a growing amount of investors (and debt, which costs interest fees).
If a FOSS project reaches maximum capacity, they will close subscriptions, they will throttle traffic, i.e. they will slow down growth, but they will not go into debt. Slowing down growth is something that a for-profit company would never do (at least until the interest rates were low and the investors were plenty, today idk). Eventually someone else in the community will decide to do a generous donation or open their own instance.
Imo that the private sector has the label of efficient is something only themselves have pushed and subsequently stuck as common knowledge. As far as I can see, the private sector is equally ridden with inefficiency.
First of all, the main contributors to a FOSS project do it for passion and do not take a salary.
This becomes more clear when you call it the Free Software movement instead of Open Source. It’s a movement that focuses on user freedom and that’s why it was created. Open Source doesn’t share the same values.
Very well put
I don’t get it. Why would you call that more efficient? In your example a profit driven company will grow at a higher rate than a FOSS project right ? So in what way is it more efficient?
Yeah your issue is youre looking it from the perspective of a cancer cell. Growth isnt always good. Too much growth and you run out of resources. Keeping things sustainable and self sufficient and not reliant on loans and “infinite growth” ponzi economics tends to work better in the long run. (Example: libraries)
i think they mean efficient in terms of resource usage, not growth.
Honestly I thought they meant resource efficient, cause we don’t need to track the users every swipe (plus another million fingerprinting methods), which saves a heck load of clocks. There’s also almost no per user saved data minus the most basic subscriptions, so everything just runs better
By not all ending up on the same server.
Could we create an onboarding system that randomly picks an instance for each new user? It is not too important which instance a user joins due to federation, right?
The instance a user joins is quite important. An instance that doesn’t want to store images and video will not want users who subscribe to all the image/video communities (that will federate their content over). A user whose interests are overwhelmingly technical won’t be interested in local communities on an artist server, where a non-technical user might feel at home. Many instance moderation policies are friendly to right-wing and will be defederated by mainstream instances. And then there are loli/shota/koda instances…
I see. Of course. That would mean the onboarding effort would have to ask a bunch of questions and each of the target instances would have to be able to describe their wants and don’t wants. This could get much too complicated I guess.
It’s certainly a challenge. I run three instances on the #fediverse. Two small ones and a larger one with 450 users. I have a donation page; initially people were enthusiastic and I covered costs. Now, I have regular monthly donators but not enough to cover costs so I am subsidising it. I took the decision when I launched that it could happen and it’s my problem.
I think there will be many instances will fall away in the coming months due to costs. Especially if you are thousands of users and associated costs.
We need to come up with a new funding model, where people appreciate you get nothing for nothing. All the large corporates sell your data as advertising for revenue. The greater public do not appreciate they are selling their soul.
Is there an approximate specs per number of users guide to size a lemmy instance?
I haven’t seen one yet. Disk usage this morning on lemmy.world was reported at about 4GB over 11 days (probably low usage). The 100GB drive would probably fill up in 275 days or so if usage did not increment. If it’s not redundant and dies, all that content is lost.
So storage will be a huge issue for lemmy unless I’m missing something.
100GB is practically nothing nowadays.
There are people (myself included, not to brag) running home servers with literally hundreds of terabytes of data. At that ~0.3 GB/day number, I alone could host 3,500 years worth of data. Get some of those r/DataHoarders and r/HomeLab guys on here and Lemmy would never run out of space.
Considering Lemmy is absolutely tiny compared to Reddit, these aren’t numbers worth considering. Every single instance needs to mirror data, and I still don’t understand how this is supposed to scale to something a fraction the size of Reddit unless the federation is just a few enormous instances that can afford that scale. It’s not like everyone pitches in what they can – every single instance individually needs to be able to support the entire dataset and the associated synchronization traffic (for the portion that its users have requested access to).
What if there are a couple large archive mirrors and the posts on other servers have a life expectancy maybe based on time, but also engagement? Crucial posts could be stickied, but I don’t see the need for everyone to hold onto everything forever. Even in the event of a catastrophic loss of the archives, the communities could still live on and rebuild.
Yeah, that’s a good point. I imagine there’d have to be some compromise like that for smaller instances. How often do users load up content older than say a couple of weeks or a month? Could be a hinderance on the experience, hard for me to estimate (for myself) how often that happens.
That is just one instance and that is a small amount of users. Reddit has 430 million active monthly users. Let’s say 1% move over to Lemmy. According to the 4 GB per 22 days (with ~1k users per instance) for Lemmy.world, that would mean you would need 1.6 Terrabytes of storage per day to support that 1% of users. Of course this would be spread over a number of instances… but you can start to see where the problem lies…
Can content be stored somewhere like S3 instead of spinning disk? It would certainly be more robust and cheaper.
Yeah, I’d love to get an approximate sense of how much these instances cost
As everyone else has already said that’s a very good question, one that doesn’t necessarily have an answer, but Im not too concerned.
I’d point out (rather excitedly) that this really isn’t unlike how the Internet used to be up until the late 00s or very early 2010s and the rise of insta, FB, birdsite, digg and reddit. EVERYone had to shoulder hosting costs (unless you were on Geocities,Myspace then it was ads)
Yes, we’ve had bulletin boards and discussion forums since perl and CGI were a thing; each was self hosted at the hoster’s expense. Newsgroup and IRC servers too - THOSE all acted like “federated” instances - common newsgroups and chat channels would be synchronized and replicated from server to server EXACTLY how federated Lemmy/Kbin/etc. instances do it now.
And the infrastructure costs were a struggle then and they will be now. Back then to have a capable CGI forum host, or to colocate your server in someone’s data center it cost a lot - like decent hosting/co-loc plans started at $50/month and went up from there. Most hosting plans had steep bandwidth caps, think like 5GB included and +$5 per GB - if you hosted a popular site 40-50GB of traffic wasn’t abnormal. If you ran a newsgroup server you frequently had to futz with how long newsgroup msgs were retained to save disk space; like 48 hrs or less (then the data would be purged).
What you can get for $50/month THESE days is quite a lot more capable, and you can run a low retention instance for a lot less. Bandwidth and disk space are ludicrously cheap (at least compared to 10-15+ yrs ago). If your instance is low user, low community, and reasonable data retention/cloning, you could run Lemmy or a Mastodon or Calkey server on an old computer you have kicking around and host it from your home internet connection with a dynamic DNS mapping.
Obviously the big instances with gobs of users will struggle with how they pay for the server infrastructure - some will use crowdfunding, patrons, donations etc. Others will run ads, or subscriptions.
My home instance lemmy.ca is at 1400 users (as of right now) and is on a $25-30/month hosting plan and so far the site is doing just fine (or seems to be). I’d guess that a massive instance like lemmy.ml might be north of $1-200. But, if you think about it, all you need are 20 ppl to donate $10/month. I donate yearly to Wikipedia. As they discuss in this thread here https://lemmy.ca/post/599590 Mastodon gets $28k Euros a month in donations and pays for two? full time developers, so its not like there aren’t people donating to open source projects… and so far Fediverse servers are doing fine.
Do you happen to know what service that’s with? Trying to see what resources that takes. ($25-$30 can mean very different specs depending on where it is hosted). Ty.
Dunno. Hey @smorks@[email protected] , what is lemmy.ca’s host provider/plan (unless its top secret Canadian Moose Power Secrets)?
it’s currently running on a $14 USD/month 4 CPU 2GB plan, but i’m going to bump it up to the $28/month 6 CPU 4 GB plan.
I suspect reddit’s reported uprofitablity isn’t due to the cost of hosting, but from blowing money in other ways.
I read they have 2000 staff members, why so many? The moderation is done by volunteers, just seems excessive.
why so many?
Engineers to develop chat and new ad features.
Sales & customer support (for advertisers, not users).
PR & media relations.
Executive junket planning & goat molestation.… yeah, no idea.
Somehow not fixing the underscore bug that breaks most links to Wikipedia.
deleted by creator
We don’t even know for a fact if they are truly unprofitable or not, it’s not like anyone here has reviewed their books.
No, but it would be extra stupid for Spez to say that if it weren’t true because it could affect investments and draw legal action.
Honestly, who knows at this point? I’ve seen some horrible business/legal decisions happen over the last 5-10 years. Some people will practically set themselves on fire just for the chance to make higher profit. Hypothetically, this certainly wouldn’t be the first case of a sketchy business drawing bad legal attention to itself, not by a long shot. I have seen a lot of businesses shut because of this type of behaviour.
The other lies from Spez about the developers certainly don’t help his case, either. That’s another fantastic way for Reddit to open themselves up to potential legal issues.
Back in the day, Reddit used to show how much of their server costs were being covered by Reddit gold revenue alone. It was basically always enough to cover daily usage.
Their hosting costs also rose exponentially when they decided to host their own videos and pictures.
Which they could have locked behind Reddit Gold or some pay-tier. /shrug
A small cloud server + a domain name costs less than a Netflix subscription. The developers have taken care to package lemmy in ways that are relatively straight forward to deploy, so a dedicated person with a small amount of experience can have an instance up and running in an evening. As long as a few percentage of users are willing to pay a netflix subscription to keep a server running, the financial burden would be spread.
I think this underestimates how users will naturally gravitate towards more centralized instances, or they’ll give up because the bigger instances are closed. Someone’s gotta pay for it, and it’s going to cost more than a Netflix subscription. Servers aren’t cheap.
This also ignores that the system isn’t horizontally scalable at all, so scaling up gets even more expensive
I think this underestimates how users will naturally gravitate towards more centralized instances, or they’ll give up because the bigger instances are closed.
(This is purely my personal opinion, of course!) In the scenario in which a few large instances dominate, the idea of the fediverse failed. One may estimate the likelyhood of success or failure given how they expect humans to behave, but in the end experiment beats theory. I think that for the fediverse to work a significant cultural shift has to occur, but I don’t think that it is an impossible shift. I would like the fediverse to succeed, and so I choose to take part in the experiment.
This also ignores that the system isn’t horizontally scalable at all, so scaling up gets even more expensive
Yes, that might cause some serious issues. The project is still in an early-development phase, and I don’t understand the technical aspects well enough yet to be able to identify whether there is obviously a fundamentally invincible barrier when it comes to scalability. My optimistic hope is that the developers are able to optimize horizontal scalability fast enough to meet the demand for scale. If it turns out to be impossible to scale, then only rich enough parties would be able to have viable instances, and that could be a reason for failure.
I think that having few big instances is not failing, it is natural for social network (where lemmy is some representation of one) to be scale-free network, which has big hubs and buch of smaller nodes connected.
Most people would go to general instances, but artists will probably go to some art focused instabce, developer to proggraming.dev… But we will have bih hubs, there is no way around it.
What does ‘horizontally scalable’ mean here? I haven’t come across that before.
This is what I think, but if anyone understands it differently please correct me.
Vertical scalability refers to scaling within a single instance. More users join and they post more content, increasing the amount of disk space needed to hold that memory, network bandwidth to handle many users downloading comments and images at once, and processing power.
Horizontal scaling refers to the lemmyverse growing because of the addition of new instances. The problem in this form of scaling is due to the resources that an instance has to use due to its interactions with other instances. So, you may create a small instance without a lot of users, but the instance might still need a lot of resources if it attempts to retrieve a lot of information (posts, comments, user information, etc) from the other larger instances. For example, at some point a community in lemmy.ml might be so popular that subscribing to that community from a small instance would be too much of a burden on the smaller instance because of the amount of memory required to save the constant stream of new posts. The horizontal scaling is a problem when the lemmyverse becomes so large that a machine with only a small amount of resources is no longer able to be part of the lemmyverse because its memory gets filled up in a few hours or days.
You can summarize by thinking of vertical scaling as “make machine bigger / more powerful” with horizontal scaling as “make more machines”.
Kind of like building a very large/tall building vs having multiple buildings!
I don’t believe this is how it works though.
Let’s say your tiny 3 person instance is connected to a big one. I believe it only pulls in content from the communities somebody from the small instance is subscribed too. Correct me if I’m wrong.
That’s what they’re saying.
Essentially - if someone from the small instance subscribes to a community that has a ton of data (huge post volume, images, whatever), the small instance needs to pull data over from the larger instance. At some point there may be communities that are so large small instances can’t pull them in without tanking.
Could that be solved by caching? Can’t the smaller instance avoid some duplication?
I wonder if there is a way to get around this? maybe smaller instances will have to be text-only?
maybe I phrased that poorly and you didn’t understand what I was trying to say. The size of the bigger instance shouldn’t matter at all because only data from communities is pulled, that a member of the smaller instance is subscribed to. So if the bigger instance has 1000 members or 2 million members wouldn’t make a difference. The only thing relevant should be how active the communities are that members are subscribed to.
That’s what I’ve gathered, but I don’t believe there’s a way for instance owners to limit what’s fetched - a user crafts the query and the server does the needful.
I imagine this could amount to a denial of service attack of sorts, if some high-churn communities are imported into tiny instances. How bad that could be, I have no idea - I’m speaking pretty theoretically, here. Text is tiny, after all, so it’s probably not much of a concern, since most of the media is actually handled elsewhere…
I’m not a web developer. I’m sort of a sysadmin so i have some experiences maintaining machines for web apps for other people. And you are right…text will not create massive amounts of data. But a lot of tiny transactions can bring down machines surprisingly fast even if the total amount of data is relatively small.
I guess we are here to experience it first hand. I don’t think anybody…not even the developers have a clear idea of how well this will scale. There is only one way to find out lol
Interesting, so would the smaller instance in this case have to perpetually store all content from the remote community, or does it just store the most recent X posts with the rest archived on the instance hosting the community? Or is it more an issue of the resources required to handle the transactions rather than the amount of data per se?
Some things can go faster if you add more workers, some things can only go faster if you make the workers bigger or faster .
If you’re tidying a garden you can get it all done more quickly, and tackle bigger gardens, by getting your friends to help. That’s horizontal scaling.
If you need to get a parcel from your house to Burkina Faso the only way to do that more quickly is to use a bigger, faster machine. That’s vertical scaling.
The way Lemmy is designed right now (says the op, I don’t know the detail) you can only support more users by making the server bigger and more expensive, not by using lots of smaller servers.
Edit: note that Lemmy as a whole scales horizontally: more instances == more users, but each instance has to scale vertically.
Put up a yearly donation drive (like Wikipedia) but unlike Wikipedia do:
- a competition between the various instances, on which collects the most donations
- not shift the page content when displaying the donation banner!
Ideally the donations will be handled through a non-profit org dedicated to this particular purpose. If the donation level is high enough, developers can be hired to further improve the source code. Currently the funds are managed through OpenCollective, but with enough growth this may not be feasible any longer.
This will most likely lead to heated debates as this will build a somewhat centralized organization, which necessarily comes with power concentration.
I’m alright with centralisation personally, as long as it’s easy enough to move off of. If Lemmy or Kbin or whatever starts being evil, you could always branch off your own organisation and still have the same content fairly seamlessly through federation. (And yes, planning to remove federation counts as evil and hopefully would cause an exodus beforehand)
The good thing is: Free development can an does happen (see Linux!). Hosting seems to be the main issue. An ideal solution would be each instance having a capacity limiter, which automatically redirects a % of content to other instances if it becomes to much. Is this possible?
That’s a bit complicated. Account handling is by nature of federation, linked to your source instance. There is discussion on GitHub about moving account export up the chain just to help people move from lemmy.ml to other instances.
Keep that discussion pinned and if you want you can see if you can help with a PR if you’re able.
a competition between the various instances, on which collects the most donations
Can you elaborate? I have the impression, that we need to think more deeply about how the donations should be distributed. E.g. a users fund are donated proportional to their subscribed Communities? I think it’s difficult, as people’s time spent on a community doesn’t necessarily mean it’s proportionally valuable. I’ve had a few subreddits which I used rarely but we’re quite important to me.
Each instance is free to field their own donation drives for their running cost. They even can display advertisements if they feel like it. There is no “one size fits all” here, and there shouldn’t.
Each instance is potentially in a different jurisdiction, making it hard to transfer money, etc.
Not only that, but I think having funds centrally collected and then distributed is a particularly bad idea. It comes with too much opportunities for bad blood. Money and friendship don’t mix.
The only unifying constant of the network is the software that runs it. This though needs to be improved in various areas, for which centrally collected funds would be ideal, as every instance will benefit from it. No operator of any instance would have a disadvantage from advertising the central donation drive. They would benefit from it by having better software in the end.
Maybe they could place some sort of incentive for donations? My instant not-seriously-scrutinized thought would be to give perhaps some sort of badge or distinction if you make a donation and maybe another sort of distinction for the top 3 donators although I’m not thinking about any possible consequences such a system might have.
Yeah, badges of distinction are one of the reddit things that Lemmy is trying to get away from.
This system is closer to old message boards than Reddit.
What the hexbear instance has done is include a donation tally in the footer and occasionally promote featured posts calling for donations when they’re needed.
Don’t forget to make a donation to your instance if you love it. For most of us it’s a bit early but I give my 10 EUR per year to my Mastodon admin. Also, if you can choose instance ran by a non profit rather than a person as it ease the whole donation mechanism and give you the right to check where your membership due go.
Is there a list of non-profits that run an instance?
So far the only one I know of is the Mastodon server Mozilla.social.
Reddit has over 2,000 employees most of whom are doing bullshit nobody using the site actually needs or wants, it’s possible to run a lot leaner than that. Like Reddit itself used to, before they started burning hundreds of millions trying to compete with every other social media site at once instead of being Reddit
bullshit nobody using the site actually needs or wants
Like being CEO.
In all honesty, there are a ton of us tech enthusiasts who have no problem paying 10-20$ per month to run an instance out of our own pockets. We get the ability to subscribe to content we used to use Reddit for, and we can have a few folks hop on with us. Multiply that by a bunch, and add in community funded instances, and we’ll be fine.
Gotta consider server costs were only a fraction of Reddit’s costs. Salaries are quite pricey, and we have lots of folks volunteering time which will make it all work.
I think the biggest cost will be image/video storage. The text takes very little space in today’s standards. The good thing is that symmetric fibre internet connections are becoming more common so it may be possible for members of the instance to contribute unused disk space to help with its image/video storage. This plus limiting the image/video sizes (and maybe forbidding video uploads altogether) will allow the instances to scale with user count.
this works on the same principals as fidonet, UseNet, email, etc. These protocols are more like fundamental services. The idea behind these was that instead of running a bunch if proprietary garbage you would run things that support A LOT of standard protocols. Why? Because NO ONE should be allowed to own our communications but ourselves.
The corps did not build these networks, we did. Software will improve over time, OSS shows the way.
Wikipedia is the 7th most visited website in the world, more popular than Amazon, TikTok, even PornHub. It’s not funded by advertisers or other bullshit - rather through reader donations.
With that said, Wikipedia is still centralized content whereas Lemmy isn’t. Meaning there’s fewer expenses and pressure on any one instance or server to succeed. And if one instance or server doesn’t succeed, your access to the Federation is far from over.
Wikipedia is set up as a nonprofit. They have annual fundraising drives asking their users for money. They also have an endowment and receive grants.
A donation drive could be a good model but the decentralized nature of the platform would complicate things.
It doesn’t have to be that complicated. Your server’s admin ask for money, setup a method to receive donations (ex: Liberapay, Paypal, etc) and there you go.
If you raise more than you need, the community can vote to donate it to other communities in need, to the Lemmy or Kbin devs or to some charity.
It doesn’t have to be complicated. It can be patreon pages for servers & instances you support, which is enough to keep the lights on. Especially if it unlocks a little cosmetic token or icon.
Removed by mod
If it is not funded through user donations, how is it self sufficient? Genuinely curious.
Idk what he’s referring to, but wikimedia funds the operating costs of Wikipedia as well as the salaries etc of those that work to keep the site running.
They have been criticised for bloat, but the site itself is entirely dependent upon donations.
WP has a story that sums it up nicely: https://www.washingtonpost.com/news/the-intersect/wp/2015/12/02/wikipedia-has-a-ton-of-money-so-why-is-it-begging-you-to-donate-yours/
What happens to your account on a federated server if that one fails though?
Then you need a new account I think. It’s a limitation of the ActivityPub protocol I think (but I haven’t done any reading). Your identity is tied to the instance it was created on.
As someone who burned reddit accts regularly this doesn’t really concern me. But if it really worries you couldn’t you set up your own private instance with you as the sole user? Nothing is more reliable than yourself. Even corporations with millions of dollars can close up shop at a moments notice.
That’s why I’m running my own server. Mastodon is much bigger than Lemmy and it does fine with community run servers.
Yeah technical users and people who are friends with technical users can just use a micro instance.
Yep, I’m using lemmy.world mostly as a trial period, but I plan on getting on my own instance to help reduce the load on the community any way I can. Will probably get a few friends on it as well.
The distributed nature of Lemmy should make things more manageable. Personally, I’m running an instance on a dedicated machine I already pay for, so it’s not costing me anything unless storage skyrockets. Many other instance hosts are also hobbyists that don’t mind covering the costs, and may take some form of donations locally on their sidebars.
There probably should be a built-in feature for instance admins to enable a local donation button to contribute to their costs, though. While Lemmy is fairly resource-efficient, larger instances are eventually going to require pretty beefy VMs to keep up with the traffic, image uploads, etc.
In general, Fedi admins simply close registrations when they can’t keep up with an influx of new users, and point people to other, smaller instances
Closing registrations is all well and good, but can’t activity / load still skyrocket as users from federated instances subscribe to, comment on, and post to their communities?
It will, but Lemmy is fairly light, so I doubt it will be critical. Fedi is about community, so in case of load problems people will crowd fund existing servers and host new ones
Can I ask about the server load? I have a Plex server sitting around doing mostly nothing. I don’t want to compromise that experience, but I’ve been thinking about starting an instance.
I was thinking about setting up a Pi to host a small/personal Lemmy instance, do you think that’s a reasonable plan? I have no clue how resource intensive Lemmy is. Was thinking it could be nice to store my and friends’ personal data on our own server instead of some random remote server (to some extent obvi).
Yes it should run perfectly fine on a Pi, at least for a small instance. You will need to get ports forwarded or setup a reverse proxy if hosting at home, since you’ll need to generate a valid SSL cert (i.e. Letsencrypt) to be able to connect to the federation
This is pretty much my exact same situation as well, plus I get so few opportunities to “pay it forward” so to speak, and now is finally my chance to do so.
