Matrix is probably something worth looking at, at least from an intellectual standpoint, for you.
It uses shared message state and a DAG, plus some fancy perfect forward secrecy (using Signal’s Double Ratchet algorithm), which is at least interesting.
There’s also Tox (chat/protocol) if you want totally distributed chat.
Personally, I really like distributed models from a theoretical standpoint; but for end-user applications they pose very difficult constraints, we live in a world with ⪅50% publicly routed IP for one, they fundamentally require immense data replication, latency in peer-finding, bandwidth constraints, and ultimately sub-par UX. I thought IPFS with a way to pay nodes to pin content was a really neat idea, but hasn’t caught on, for example. Not to discourage you, if you think it’s workable then have at it, but I think it at least explains the current state of things.
Yeah, I’ll have to look into Matrix’s design. I know some of the basics, but the devil is in the details.
we live in a world with ⪅50% publicly routed IP for one
STUN servers bring access up dramatically, and there’s always TURN for the stragglers. Things seem to be getting better with more and more people getting IPv6. I still don’t have it though, and I’m also behind CGNAT, so I know the pain.
But I don’t think bandwidth is really a problem. You should use similar bandwidth to a centralized service, provided the application does appropriate caching and there’s some form of cooperative querying to reduce wire transfer.
I’m following Iroh development (started as a faster replacement for IPFS), and this video was particularly instructive for reducing wire overhead (fancy bloom filter application). I’m trying to build something like Lemmy with it, and I’m interested to see if a similar approach could work for something like Discord.
But yeah, given how much I’m struggling with it certainly explains why it’s not very popular. I could build my project as a centralized service in much less time because synchronizing between clients is very straightforward, and something I do every day at work. But we already have that, and the Fediverse just takes that idea and connects nodes together, so I wouldn’t be adding anything. However, I think longer term, something like this (probably not my specific project) is going to be really important, and I think it’ll be “fast enough.”
pay nodes to pin content
Yeah, I don’t think that’s going to work, because you’d have to pay a sufficient number of nodes such that one node losing interest won’t screw you. And then you need some form of contract (smart contracts on a blockchain?) to ensure you are getting what you’re paying for instead of just getting screwed by someone tossing the data and never telling you.
I’m trying to design my system such that everyone participates in some small way in supporting the network. If you’re on a phone, you store a lot less than if you’re on a PC, but more than if you’re in a web browser. Maybe we could have a distributed fund for rewarding people for supplying more resources than necessary, idk, but I’m honestly not interested in the money part, I just want to build something cool that can’t be taken down because someone got bought out, lost interest, or their government disconnected from the internet. In fact, my system should “just work” if you have people traveling w/ data between isolated networks, provided they have a list of relays within each network that they can connect to peers with.
Matrix is probably something worth looking at, at least from an intellectual standpoint, for you. It uses shared message state and a DAG, plus some fancy perfect forward secrecy (using Signal’s Double Ratchet algorithm), which is at least interesting. There’s also Tox (chat/protocol) if you want totally distributed chat.
Personally, I really like distributed models from a theoretical standpoint; but for end-user applications they pose very difficult constraints, we live in a world with ⪅50% publicly routed IP for one, they fundamentally require immense data replication, latency in peer-finding, bandwidth constraints, and ultimately sub-par UX. I thought IPFS with a way to pay nodes to pin content was a really neat idea, but hasn’t caught on, for example. Not to discourage you, if you think it’s workable then have at it, but I think it at least explains the current state of things.
Yeah, I’ll have to look into Matrix’s design. I know some of the basics, but the devil is in the details.
STUN servers bring access up dramatically, and there’s always TURN for the stragglers. Things seem to be getting better with more and more people getting IPv6. I still don’t have it though, and I’m also behind CGNAT, so I know the pain.
But I don’t think bandwidth is really a problem. You should use similar bandwidth to a centralized service, provided the application does appropriate caching and there’s some form of cooperative querying to reduce wire transfer.
I’m following Iroh development (started as a faster replacement for IPFS), and this video was particularly instructive for reducing wire overhead (fancy bloom filter application). I’m trying to build something like Lemmy with it, and I’m interested to see if a similar approach could work for something like Discord.
But yeah, given how much I’m struggling with it certainly explains why it’s not very popular. I could build my project as a centralized service in much less time because synchronizing between clients is very straightforward, and something I do every day at work. But we already have that, and the Fediverse just takes that idea and connects nodes together, so I wouldn’t be adding anything. However, I think longer term, something like this (probably not my specific project) is going to be really important, and I think it’ll be “fast enough.”
Yeah, I don’t think that’s going to work, because you’d have to pay a sufficient number of nodes such that one node losing interest won’t screw you. And then you need some form of contract (smart contracts on a blockchain?) to ensure you are getting what you’re paying for instead of just getting screwed by someone tossing the data and never telling you.
I’m trying to design my system such that everyone participates in some small way in supporting the network. If you’re on a phone, you store a lot less than if you’re on a PC, but more than if you’re in a web browser. Maybe we could have a distributed fund for rewarding people for supplying more resources than necessary, idk, but I’m honestly not interested in the money part, I just want to build something cool that can’t be taken down because someone got bought out, lost interest, or their government disconnected from the internet. In fact, my system should “just work” if you have people traveling w/ data between isolated networks, provided they have a list of relays within each network that they can connect to peers with.