Why YSK: Because intuitive explanations are few and far between, and the technical explanations often present too many “trees” and not enough “forest”, which is just how technically-minded people are trained to approach things. Forests are, after all, made of trees, and it’s not their fault we don’t care about individual “trees”. This then, is my unprofessional attempt to consolidate everything I’ve read over the past three days into one, easy-to-understand explanation of how all this shit works in lay-person’s terms. Due to my amateur background, I may have details incorrect, and I would request that anyone who catches anywhere where I have made a mistake, even a small detail, to please correct me. I will also include a few links to my best sources at the bottom. tl;dr style explanations will be included after every paragraph in parenthesis. So, let’s begin:
Imagine you have reddit. Fantastic, it’s a giant forum composed of a whole bunch of smaller, sub-forums. But let’s take this one step further. Why have just one reddit? Why can’t we have lots of reddits, each capable of having its own complete set of subs, where each reddit is independent of every other one and has its own web address? Okay, let’s do this, and push it to the extreme. Let’s make it so everyone can make their own reddit, even individuals. So you, if you wanted, could set up your own complete reddit, with just you in it. You could have all the subs, r/TIL, r/TIHI, r/pics, etc etc, all with just you in them. You have total control! But you have no content and are probably pretty lonely, right? We’ll get to that. Let’s call this Self-Hosting though.
(So, we now have a situation where many whole reddits can individually exist, each in the vacuum of space.)
Now let’s fix that content and loneliness problem. What if we allowed each reddit to communicate and share content with every other reddit, similar to how subs can communicate with each other? Boom. We just created a spider-webbed network, of countless individual reddits, each composed of subs, that can now all share content back and forth. Let’s call this big spiderweb an over-reddit, to contrast it with subreddits.
(Now instead of a two-tier system of isolated reddits and their subs, we have a three-tier system, of over-reddit [the “Lemmy-verse”], reddits [Lemmys or Instances or Servers], and subs [communities or sub-lemmies].)
But, we actually have a technical problem. How do these individual reddits find each other? How do they know the other ones even exist? They could be on servers on opposite sides of the planet, with random web addresses. Obviously we can’t just guess. So, okay, let’s let users solve this for us via crowd-sourced labor. We don’t have to find all the reddits for them. Let’s just design the system so that the reddits only find out about each other after any random-ass user introduces them to each other. We’ll call this batching, they can do it with the reddit search bar. Then, we’ll wait for that random-ass user to actually subscribe to any new sub/community over there, which they’ll only do if it’s any good. Once this is done, now the two reddits and that one sub become connected, not just for that user, but their whole reddit userbase too. The rando doing the search and subscribing simply introduced two good reddits to each other. Now that they know about each other though, they’ll share content back and forth freely, with comments, votes and posts all being visible to both reddits. Let’s call this “federating with each other”. It’s not too different from neurons in the brain reaching out to each other, really.
(To find and connect the disparate, scattered reddits into our over-reddit, we use crowd-sourced labor.)
Well, that’s it. That’s the Lemmy-verse. But what about this Fediverse? Well, okay, remember what we did with reddit, and giving it a third tier of over-reddit? Let’s do the same exact thing with twitter, facebook, youtube and every other thing we can pull out of our asses. Let’s let all of them share and access each other’s content with the exact same structure and system, so now you, hanging out in your reddit, can get all the tweets too. We’ve made a fourth tier now. The Fediverse, which is most comparable to the internet itself, and includes the Mastadon-verse, the PeerTube-verse, etc etc.
(Why stop there…? reddit is chump change, let’s just do this to everything.)
So, that’s it in a nutshell. That’s how this shit works. And the next time someone says it’s like email, I’m going to climb through their computer screen and smack them. It’s only like email at that technical, “trees” level, and when you go up to the more intuitive “forest” level, this just serves to confuse the ever-living hell out of everyone.
(I’m a bit of a dick.)
One last detail: Admins can whitelist (allow-list) or blacklist (shadowban) other Instances/servers. As an example, one of the other largest Instances has blacklisted (shadowbanned) us here at lemmy.world, because we were producing too much spam. As a result, until they undo this, all of us here are shadowbanned from their Instance/server. We can see their content, they can’t see ours. This enables them to control how much connection they have to the rest of the Fediverse.
(Let’s not forget to give admins the power to stop people from other places bothering them, if they do not approve of the content. Very important feature.)
Sources: https://join-lemmy.org/docs/users/01-getting-started.html https://en.wikipedia.org/wiki/Lemmy_(software) https://github.com/amirzaidi/lemmy https://news.ycombinator.com/item?id=36387939
Again, if I’ve made any errors, regardless of how small, please let me know below. This is intended to be another reference material for lay-people, so accuracy is important. However, outside of major errors, I will not be editing this post to correct it, as I would prefer any corrections to be delivered from the full perspective of someone’s individual expertise, instead of being translated into my own words.
(I don’t actually know what I’m talking about. Scroll down for people who do.)
Hope this helps.
edit for grammar/cleanup
Definitely a nicely written explanation. Though I still feel it will be too much for the general population to understand and adopt.
Understanding is not required for adoption. How many redditors really actually know how it works?
I think the problem is that you have to have at least some understanding of how the federation aspect functions to make sense of Lemmy. I’d love to contribute to their UX, there’s a lot of room for improvement IMO.
I’m definitely hoping a lot of technical solutions to current problems will be arriving in the coming months.
I have a basic question I’m hoping someone can answer. I want to read this post, but when I follow the link, it takes me to a Game Thread every time., but with the comment section of the YSK post. Even when I go to Candelestine’s profile and click on the post, I instead get some Game Thread (last time it was Braves @Reds) with comments about Lemmy and federation. What am I doing wrong?
That is a bug. Something something websockets something next update. Maybe.
It’s an inconsistent bug. If you keep trying you will get through eventually.
At least it’s not me! Thanks, and bummer.
Yeah there’s like at least a half a dozen mildlyinfuriating and nostupidquestions threads about it. Probably more by now. They tend to be pretty amusing.
I see several things being brought up a lot on Reddit about it. Folks seem to be horrified by the idea that more than one instance could have a /c/technology.
But there are some legitimate questions about what’s different about the experience because federation. Like, can one instance block a user from another instance? If I make posts (like this one) on foreign instances, then delete my account on my home instance, what happens to this post? If astronaut makes bafroom in his pants, does he go boom?
Yeah, unless one of the fediverse sites can simplify I don’t see it taking off among regular people.
I think the search just needs to get better. The large instances should already have connections to the most popular instances and communities. So the average person isn’t likely going to need to be doing the linking. But the search right now just feels clunky
So i may have understood wrong here but i think the analogy is like saying something like:
instead of reddit being a central hub with “rooms” (subreddits) branching off, we instead have multiple reddit hubs which have their own sets of rooms. But a lot of those rooms have little back doors or hallways to the same room at the other reddits, so the users in a particular room can see eveything in all the other alternate version of the same room as if it was the same one.
Like being in the same subreddit without realising its a compilation of contant from all the other rooms across the lemmyverse.But also that if the people in a particular room start getting rowdy or dangerous then the other rooms can lock their backdoor connecting the rooms so the unruly users cant come in from that other room until they decide to unlock the door again.
Have i got this all right?
Good. Getting “regular people” to use reddit is what ruined it. Let them keep reddit and we can stick to the fediverse.
Yeah, it certainly lost its charm when comment sections became more like Facebook’s/YouTube’s/Your-local-newspapers-website’s.
You’re not wrong, but Mastodon’s already made the Fediverse somewhat ‘normiefied’.
Then, we’ll wait for that random-ass user to actually subscribe to any new sub/community over there, which they’ll only do if it’s any good
[citation needed]
Nice. Can you explain this using poop diagrams?
Maybe you have to wait 3 days to get an answer.
This is a great writeup, but if someone could elaborate on how a piece of content in the Lemmyverse gets into the Mastadon-verse, and how interaction on that piece of content in both -verses is handled across the verses, I would appreciate it.
From what I’ve seen, it sounds like since everything in the fediverse shares the ActivityPub APIs under the hood, it’s possible to share the same piece of content across verses. But I can’t wrap my head around how interactions with the content across -verses is possible.
Imagine a UN gathering with government officials of different countries speaking English as common language, simultaneously CNN of each country streams the translated message in local language.
Same thing happens between different Fediverse apps. Within the app, the info is stored differently, but it gets mapped to the common ActivityPub fields.
You will fully grasp this concept after a quick read of this page on how does Bookwyrm interact with Mastodon:
Ahh this is great. So there’s a standard set of object types defined by ActivityPub. Verses can define their own custom types, but they can serialize out to the common types, albeit with a likely loss of functionality.
So for example, you could post a pieces of content to Lemmy, it can be visible on Mastodon. Users from both could comment on it, and the comments would be seen across both verses. But the Mastadon users might not see something like the up/down votes?
So for example, you could post a pieces of content to Lemmy, it can be visible on Mastodon. Users from both could comment on it, and the comments would be seen across both verses. But the Mastadon users might not see something like the up/down votes?
Exactly !
Isn’t that amazing? I love the idea.
It’s definitely a fascinating idea - effectively you can build different UX to interact with the same content! But it is a bit brain bending to think about the implications of that interaction-wise.
Using the same example above - since Mastadon users don’t have the voting, that likely means that they see the comments in a different order than a user viewing the same piece of content using Lemmy. That’ll drive people to interact with that content in a different way. Which will have interesting outcomes. Will Mastadon-sourced comments end up making sense when viewed in Lemmy?
I’m in the same boat. The various commercial platforms have very different formats, and even ignoring that being they’re commercial entities whose business models are based on keeping you on their platform as much as possible, they very intentionally do not inter-operate. So sharing content between them is often done via posting screenshots.
How does a system that actually works…work?
Thanks for the explanation! Although I consider myself somewhat techsavy, the whole system was confusing at first and only after a couple of weeks of using it/researching/trying to understand how shit works (in parallel to uni finals) I am starting to get a hang of it. Your post really helped to get the overall picture of how this federation works!
I do recognize this post is mostly directed to redditors (include myself) but could sound more neutral not using reddit as example all the time. You can refer to a more neutral “web page”. Just a suggestion.
It helped.
Thanks for this! I think I mostly understood it from reading the technical stuff, but not fully. This helps a lot though.
Thanks! You explanation really helps.
Honestly, the ‘email’ analogy is still the best one for most people. Takes less time too, which is important when most people have an attention span of less than a femtosecond.
Honestly, I just don’t think there is a single easy analogy that actually works for most people. That one or any other one. That was why I slipped in the neurology analogy. Different analogies for different folks.
But most people think in terms of features and experience, not mechanism. So, if this was like email, it would be secure, private, one-on-one and more annoying than fun.
I dunno, whenever I use the email analogy to explain the interoperability aspect, it makes people feel more at-ease with the whole ‘federation’ thing. Most people, after all, get email.
You might be better at employing the analogy than most are then.
Perhaps. It’s worked well for me so far, though I can’t speak for others
How do you formulate it?
I basically use it to explain how accounts work: ‘It’s kinda like having a Gmail account. You have a Gmail address and can only login through Gmail or apps that support it, but you can send Emails to people with Yahoo accounts. Similarly, having a lemmy.world account will only let you login through lemmy.world, but you still have full interaction with lemmy.ml users, or even entirely different ‘platforms’ like Kbin.’
Sure, it’s extremely dumbed down and not entirely accurate, but it’s easy enough for people to relate to, and it puts them at ease about the whole ‘fediverse’ thing.
Hm. That is admittedly worded in a simpler way than I usually see, perhaps. I’ll keep it in my back pocket, thanks for sharing it.
I may have been too hasty in my dismissal of the analogy.
Thanks. I am using Memmy on TestFlight and it’s made the Lemmy experience so much easier.
One question I have though. Can I search for a kbin ’subreddit’ or whatever they are called or a Squabble sub to my Lenny profile so it brings all of them into one or so I need the account I have on each of them and treat them as 3 independent sites?
You only ever need to make one account. From that main server where your account is, you should be able to go into the search bar and punch in the URL (or a special Fediverse link, they always start with a !) of anything else on the entire Fediverse. Once there, you should be able to subscribe to that new place without having to create any new accounts. Once the process finishes, the new content should start appearing in your feed back in your home community.
Do be aware that the process is a little buggy, and here on lemmy.world via web browser, I often get “No Results” even though it actually is in the searching process and will eventually find them. It does take a few minutes though.
Righto. I feel like a graph or image of these in line with the explanation will do wonders for a visual learner. I am no design whiz but can probably do it on Apples Freeform or something. Thanks for the detailed reply.
Kbin and Lemmy can join together (federate) 😁 a kbin ‘sub’ is called a magazine. For example, you can view the kbin ‘tech’ magazine here: https://lemmy.world/c/[email protected]
To my knowledge, squabbles is not based on ActivityPub and therefore cannot federate with Lemmy.
As for searching for a kbin magazine from Lemmy, you’ve got me there and I have no idea!
Thank you kindly for the reply! I’ll keep my Lemmy account and a Squabbles one and that will do me then :)
We can see their content, they can’t see ours.
Wasn’t defederation a two-way thing? Basically the whole instance (website, in laymen terms) blocked you. Nothing goes in, nothing goes out between you two.
Not an expert, just relaying what I read elsewhere. TLDR: “nothing goes in, nothing goes out between you two” is about right, but it’s not entirely symmetrical.
Instance A defederates from instance B. This means that (let instance C be a neutral third party that is federated with both A and B):
- no posts or comments by users from B anywhere on lemmy are visible to users from A
- existing posts and comments in communities on A are still visible to users from B
- new posts and comments by users from A and C in communities on A are not visible to users from B
- new comments by users from B to existing posts in communities on A are not visible to users from C
- new and existing posts and comments by users from A in communities on C are still visible to users from B
Thank you for asking this. I believe it works as I described, but I myself am hoping for either a correction or a confirmation from a knowledgeable expert. My understanding mainly came from the ycombinator thread, which is not a perfectly reliable source.
Even if that’s the case, I THINK this instance can get the content transitively through the other lemmies it does federate with. I could be wrong there as well though
Thanks for this ! I learned quite a few things, even on stuff I thought I had already understood !
Ok I understood most of this already and it all is pretty neat. However, I did not know this “batching” part. So if I started up my own lemmy today with a few other subreddits underneath it, I would be able to search/find those from lemmy.world later today?
Correct.
That’s a lot of words for it’s the same concept as using any oAuth (Google signin, Facebook, etc.) Does Google control that random site that lets you use your credentials to sign up, nope it’s just an accepted method people agreed to. Replace oAuth with lemmy and each site adopting the method to talk they just can, but don’t always talk to each other too.
Welcome to really old technology, again the Usenet