This is part of the reason why e-Mail works the same now as it did in the 80s.
I still want to see a proof that there isn’t a technical solution for this.
There are things like versioned APIs, backwards compatibility… You can make your network protocol modular and extensible… Think of XMPP and some other examples.
E-Mail is somewhat alright and has a few good design choices. That’s why it’s still around today. With the additional lessons learned since then, todays knowledge and tools, I bet we can design some technical solutions to the upgradeablility-problem.
It’s absolutely just a skill issue, matrix has made breaking changes without significant issues.
Turns out that if you just design a protocol with changes in mind you can simply reserve a version namespace for all but the most fundamental functionality and crank the number up for every breaking change.
Yeah. You can tell Email is old. very old. The internet has exploded since then. There are so many more nodes and users out there than anyone would have imagined in the early 80s. Also technology has advanced together with how we use it. 7-bit is madness by today’s standards. Of course Antispam and E2EE hasn’t been baked in because it wasn’t a thing back then.
But things are different today. I don’t think there will be another ‘explosion’ so that the requirements to such a protocol and it’s usage will change as quickly and fundamentally.
Funny thing is, the resource usage of my mailserver or XMPP server is so much less than for example my Matrix server or any of the other ‘modern’ federated things i tried… And we should learn from XMPP’s history. Both good and bad things. It’s a complicated story and there is more to the story than just network effect or technical issues. And I love and hate Matrix. I’m glad it’s there but i also wasted several hours looking for good client for linux that isn’t element and uses all of my RAM. And fought with encryption in some python libraries. Sometimes matrix just isn’t fun. Especially the encryption bit.
I don’t care for the network effect. I have used both XMPP and Matrix. There was a time I could reach all my friends via XMPP. Back in the days when both Facebook and GMail had XMPP and WhatsApp wasn’t there yet.
As of today. I use Matrix. And the few people I talk to most frequently also use Matrix. And that’s enough for me. I don’t care if 99% of other people use something different. (Also there are bridges to other protocols). It’s the same with Lemmy. I wouldn’t be here if it was important to me to be on a platform with 1.5 billion other users.
I think there is the possibility to do it right. And to design something that will last for quite a while. Of course there are issues to solve on several levels. Unfortunately back in the days, protocols were invented by scientists and to connect universities. Todays platforms are implemented by mega-corporations and their motive is to gather data and sell advertisement. So we probably need regulations and politics to force something like interoperability into existence. And of course there is the age-old question of reform vs. revolution. Iterative change sometimes isn’t good enough. I’d consider email a case where we need revolution. I’d happily use some free sucessor. Even before the network effect or regulation kicks in.
There are so many issues with email that are not fixable… And federation, dispite all the advantages it has, is the main reason why it’s entirely unfeasible to actually fix the issues in the system.
Federation, same as every other concept, has advantages and downsides.
Having looked into the Lemmy code and the discussions on Github sadly doesn’t bode too well. It looks like a mostly quick-and-dirty project and I fear it’s going to get only more troublesome as it grows. I am not sure if it could ever scale to Reddit-dimensions.
Part of that can already be observed with all the desynchronisation between instances, because there is no guaranteed eventual consistency or any other mechanisms like that, even though that would be fundamentally important for a distributed system like Lemmy.
But extensions are no good if most people don’t use them. Take end-to-end encryption in eMail. It’s a good feature that has been around for multiple decades, but most people don’t use it. Since most people don’t use it, there’s no point in using it. So you have the network effect right inside your system.
When e.g. WhatsApp made every chat end-to-end encrypted it took a single update and went so smooth and easy that most people wouldn’t have noticed if it wasn’t for a big modal telling the useres that it was introduced.
Introducing breaking changes or new features to a federated system with lots of hosts and lots of different software implementations is certainly not impossible, but it’s much more difficult than on a centrally managed system.
You could argue it’s a good thing that no entity is able to force everyone into using every new extension. But true. You then have issues with people and politics. You could just do a lookup on a keyserver and do opportunistic encryption. That wouldn’t harm anyone. (If done right.) Gmail could implement that and a major part of email users would have e2ee overnight and benefit from that.
Regarding WhatsApp. I remember shaking my head about WhatsApp when people started using it. As far as i remember (i might be wrong) It was widely open, unencrypted and everyone could impersonate anyone they had the phone number of. I don’t remember why it got so popular. But I’m glad they implemented encryption and fixed that.
With email I’m at least theoretically able to do something myself. With WhatsApps issues, there is no way to do anything about it. You just have to accept it’s quirks, because only Meta could implement something. For example I’d like to use it on my computer. And have a different identifier than my phone number. And stop it leaking metadata to Meta. How does a non-federated platform like WA help me with that?
For a new and federated protocol you could start with mandatory end to end encryption. And you then design the protocol so that changes won’t be breaking. And if you do it right it’ll be okay if people don’t adopt extensions. Things will still work. Maybe someone can’t do video calls or show emoji reactions. Maybe the cutting edge AR or VR stuff doesn’t work. But at least you have a fallback to send encrypted text data or arbitrary data-files. That should be enough.
The thing is that for some features to have any benefit you actually need everyone on board. Security is just that.
If you have to basically have a fallback-backdoor built right into your system to deal with those who don’t participate in the security system, an attacker just needs to force the fallback and nothing is secure anymore.
And sure, Gmail could just force encryption, but then (a) would everyone complain about one big actor abusing their market power, as happens a lot e.g. with Chrome and (b) the whole point of using email is that it’s a service that’s super stable and “just works”. If I can’t send an email to my dentist about an appointment, then it’s worthless. So something like that could hurt Gmail’s market share.
But all in all, my point was that open systems with lots of actors with the power to decide stuff makes implementing important changes more difficult, because you have to convince much more people to follow suit.
Yeah. I get it. You’re right. If there is only one actor, they can make decisions more easily. If there are multiple actors involved like with federated stuff, you add additional overlay by having to agree and have methods like voting, consensus etc.
My point is: It is possible. I don’t disagree that takes extra work. But we live in a democraty, not a monarchy. We have technical solutions. You keep saying we need consensus between every instance of a federated software and 100% solutions. But that simply isn’t true. We don’t need consensus. We don’t need everyone to agree. You could just expel everyone from the network that hasn’t updated their server for 3 years from the network. You won’t even notice the <1% users that go missing. You could implement text, audio, video, group chat mandatory encryption and minimize metadata. Make it performant and extensible and a backwards-compatible protocol. You might only be 95% of the way. But isn’t that better than anything currently available? It’ll probably stay that way for some time if you did it right. Just forget the last 5% to make it a theoretically perfect solution.
With the encryption: As with everything security related, it depends on your specific thread model. My example would help against everyone casually reading everyone else’s mail. It won’t help against a targeted attack IF you could force the fallback triggering and there wasn’t such a thing like certificate pinning. But it’s a thousand percent better than not doing anything at all because it could be curcumvented in an edge case. But I don’t want to argue in email’s favor. email is old. the only reasonable option is to start over. and force reasonable encryption this time.
Regarding the network effect: Nothing new is going to happen in the world if we don’t fight it. Many people are conservative. We buy the stuff we’re familiar with instead of something better. We want the things everyone has despite there being better alternatives. Americans keep using the vastly inferior imperial system. We sometimes need to get done with tasks and use that thing that is compatible with people we want to interact with. Like the messenger, the social media platform everyone uses. Microsofts office software to interact with clients… I understand. But again, there are ways around this. You could establish something nice and better in your small community and stop caring for the rest of the world. You could use something like a bridge that connects old and new technology. You can be a country and make laws that force something into existence. You can be a big corporation and just foist the the new thing on your users. Like the Instagram accounts that kickstart Threads. I don’t say it’s necessarily easy to do. But possible.
I still want to see a proof that there isn’t a technical solution for this.
There are things like versioned APIs, backwards compatibility… You can make your network protocol modular and extensible… Think of XMPP and some other examples.
E-Mail is somewhat alright and has a few good design choices. That’s why it’s still around today. With the additional lessons learned since then, todays knowledge and tools, I bet we can design some technical solutions to the upgradeablility-problem.
It’s absolutely just a skill issue, matrix has made breaking changes without significant issues.
Turns out that if you just design a protocol with changes in mind you can simply reserve a version namespace for all but the most fundamental functionality and crank the number up for every breaking change.
[This comment has been deleted by an automated system]
Yeah. You can tell Email is old. very old. The internet has exploded since then. There are so many more nodes and users out there than anyone would have imagined in the early 80s. Also technology has advanced together with how we use it. 7-bit is madness by today’s standards. Of course Antispam and E2EE hasn’t been baked in because it wasn’t a thing back then.
But things are different today. I don’t think there will be another ‘explosion’ so that the requirements to such a protocol and it’s usage will change as quickly and fundamentally.
Funny thing is, the resource usage of my mailserver or XMPP server is so much less than for example my Matrix server or any of the other ‘modern’ federated things i tried… And we should learn from XMPP’s history. Both good and bad things. It’s a complicated story and there is more to the story than just network effect or technical issues. And I love and hate Matrix. I’m glad it’s there but i also wasted several hours looking for good client for linux that isn’t element and uses all of my RAM. And fought with encryption in some python libraries. Sometimes matrix just isn’t fun. Especially the encryption bit.
I don’t care for the network effect. I have used both XMPP and Matrix. There was a time I could reach all my friends via XMPP. Back in the days when both Facebook and GMail had XMPP and WhatsApp wasn’t there yet. As of today. I use Matrix. And the few people I talk to most frequently also use Matrix. And that’s enough for me. I don’t care if 99% of other people use something different. (Also there are bridges to other protocols). It’s the same with Lemmy. I wouldn’t be here if it was important to me to be on a platform with 1.5 billion other users.
I think there is the possibility to do it right. And to design something that will last for quite a while. Of course there are issues to solve on several levels. Unfortunately back in the days, protocols were invented by scientists and to connect universities. Todays platforms are implemented by mega-corporations and their motive is to gather data and sell advertisement. So we probably need regulations and politics to force something like interoperability into existence. And of course there is the age-old question of reform vs. revolution. Iterative change sometimes isn’t good enough. I’d consider email a case where we need revolution. I’d happily use some free sucessor. Even before the network effect or regulation kicks in.
There are so many issues with email that are not fixable… And federation, dispite all the advantages it has, is the main reason why it’s entirely unfeasible to actually fix the issues in the system.
Federation, same as every other concept, has advantages and downsides.
Having looked into the Lemmy code and the discussions on Github sadly doesn’t bode too well. It looks like a mostly quick-and-dirty project and I fear it’s going to get only more troublesome as it grows. I am not sure if it could ever scale to Reddit-dimensions.
Part of that can already be observed with all the desynchronisation between instances, because there is no guaranteed eventual consistency or any other mechanisms like that, even though that would be fundamentally important for a distributed system like Lemmy.
This is exactly what I wanted to say, just put much more eloquently. Thanks!
But extensions are no good if most people don’t use them. Take end-to-end encryption in eMail. It’s a good feature that has been around for multiple decades, but most people don’t use it. Since most people don’t use it, there’s no point in using it. So you have the network effect right inside your system.
When e.g. WhatsApp made every chat end-to-end encrypted it took a single update and went so smooth and easy that most people wouldn’t have noticed if it wasn’t for a big modal telling the useres that it was introduced.
Introducing breaking changes or new features to a federated system with lots of hosts and lots of different software implementations is certainly not impossible, but it’s much more difficult than on a centrally managed system.
You could argue it’s a good thing that no entity is able to force everyone into using every new extension. But true. You then have issues with people and politics. You could just do a lookup on a keyserver and do opportunistic encryption. That wouldn’t harm anyone. (If done right.) Gmail could implement that and a major part of email users would have e2ee overnight and benefit from that.
Regarding WhatsApp. I remember shaking my head about WhatsApp when people started using it. As far as i remember (i might be wrong) It was widely open, unencrypted and everyone could impersonate anyone they had the phone number of. I don’t remember why it got so popular. But I’m glad they implemented encryption and fixed that.
With email I’m at least theoretically able to do something myself. With WhatsApps issues, there is no way to do anything about it. You just have to accept it’s quirks, because only Meta could implement something. For example I’d like to use it on my computer. And have a different identifier than my phone number. And stop it leaking metadata to Meta. How does a non-federated platform like WA help me with that?
For a new and federated protocol you could start with mandatory end to end encryption. And you then design the protocol so that changes won’t be breaking. And if you do it right it’ll be okay if people don’t adopt extensions. Things will still work. Maybe someone can’t do video calls or show emoji reactions. Maybe the cutting edge AR or VR stuff doesn’t work. But at least you have a fallback to send encrypted text data or arbitrary data-files. That should be enough.
The thing is that for some features to have any benefit you actually need everyone on board. Security is just that.
If you have to basically have a fallback-backdoor built right into your system to deal with those who don’t participate in the security system, an attacker just needs to force the fallback and nothing is secure anymore.
And sure, Gmail could just force encryption, but then (a) would everyone complain about one big actor abusing their market power, as happens a lot e.g. with Chrome and (b) the whole point of using email is that it’s a service that’s super stable and “just works”. If I can’t send an email to my dentist about an appointment, then it’s worthless. So something like that could hurt Gmail’s market share.
But all in all, my point was that open systems with lots of actors with the power to decide stuff makes implementing important changes more difficult, because you have to convince much more people to follow suit.
Yeah. I get it. You’re right. If there is only one actor, they can make decisions more easily. If there are multiple actors involved like with federated stuff, you add additional overlay by having to agree and have methods like voting, consensus etc.
My point is: It is possible. I don’t disagree that takes extra work. But we live in a democraty, not a monarchy. We have technical solutions. You keep saying we need consensus between every instance of a federated software and 100% solutions. But that simply isn’t true. We don’t need consensus. We don’t need everyone to agree. You could just expel everyone from the network that hasn’t updated their server for 3 years from the network. You won’t even notice the <1% users that go missing. You could implement text, audio, video, group chat mandatory encryption and minimize metadata. Make it performant and extensible and a backwards-compatible protocol. You might only be 95% of the way. But isn’t that better than anything currently available? It’ll probably stay that way for some time if you did it right. Just forget the last 5% to make it a theoretically perfect solution.
With the encryption: As with everything security related, it depends on your specific thread model. My example would help against everyone casually reading everyone else’s mail. It won’t help against a targeted attack IF you could force the fallback triggering and there wasn’t such a thing like certificate pinning. But it’s a thousand percent better than not doing anything at all because it could be curcumvented in an edge case. But I don’t want to argue in email’s favor. email is old. the only reasonable option is to start over. and force reasonable encryption this time.
Regarding the network effect: Nothing new is going to happen in the world if we don’t fight it. Many people are conservative. We buy the stuff we’re familiar with instead of something better. We want the things everyone has despite there being better alternatives. Americans keep using the vastly inferior imperial system. We sometimes need to get done with tasks and use that thing that is compatible with people we want to interact with. Like the messenger, the social media platform everyone uses. Microsofts office software to interact with clients… I understand. But again, there are ways around this. You could establish something nice and better in your small community and stop caring for the rest of the world. You could use something like a bridge that connects old and new technology. You can be a country and make laws that force something into existence. You can be a big corporation and just foist the the new thing on your users. Like the Instagram accounts that kickstart Threads. I don’t say it’s necessarily easy to do. But possible.