I really wish Matrix had been more successful, but it has some pretty core problems that prevented it from gaining more traction.
It fell into the same trap as XMPP, though perhaps even worse, with a focus more on its protocol and specification than a single unified product vision.
The reference server implementation is slow, and using a language not optimal for its purpose, with alternative server implementations left incomplete and unsupported.
It took a long time for them to figure out voice and video and for it to work well, and the “user flow” still isn’t at Discord levels.
I’ve rooted for Matrix for a long time, but as a former XMPP evangelist, to me the writing on the wall says it isn’t suited for success either. I’d love to be wrong, but I don’t see a way through.
In terms of performance, there’s Rust in the synapse repo already, and both Conduit (Rust) and Dendrite (Go) seem viable. If one of those projects reaches parity with Synapse, do you think that’ll “fix” matrix?
If not, are there other issues core to matrix design? I’m not that familiar with matrix except as an occasional user that follows a few tech rooms, but I’d love to help out if I’m pointed in the right direction. I’m comfortable with Rust and Go (and do Python at my day job), so if backend performance is a bottleneck, I could make help out.
But if the problems are fundamental to how it’s designed or how the project operates, I’d rather work on other things.
I do think the other home server implementations gaining parity (production-ready) with the reference home server would go a long way. I haven’t run a home server but I’ve heard from those that have that it really has a hard time scaling. (Though this serves as impetus to give it a try over spring break)
Which brings me to the caveats of the protocol, I personally don’t think the design is ideal, it’s more described as a distributed message bus, what I’ve read of the spec it’s over engineered, it made good decisions wrt using modern web technologies (JSON, WebRTC), but it didn’t scope itself to the particular task.
That said, I haven’t written a federated protocol, and they have.
But if I was going to, I’d really want to look at Discord and see how to copy a lot of that model, but break parts of it out to facilitate federation:
I originally wrote a huge hypothetical design here that I speculated would fare better, but honestly the specifics become less relevant, point is that the shared state of rooms is a real challenge, and one out of scope for just a federated instant messaging system, and I’m no longer certain it’s viable.
I’m personally more interested in P2P protocols than federated, so that’s the stuff I build in my spare time.
So instead of something like Lemmy or Matrix, I’d have something like BitTorrent or Tor, so nodes just add capacity instead of hosting specific content. You could configure your node(s) to pin specific content (e.g. for backups or latency), but your data would also be distributed to other peoples’ computers.
This provides data redundancy, permanency of the service (no centralization whatsoever), and ease of scaling (every client could store and seed data), but comes with complexity. I think it’s workable though.
Matrix is probably something worth looking at, at least from an intellectual standpoint, for you.
It uses shared message state and a DAG, plus some fancy perfect forward secrecy (using Signal’s Double Ratchet algorithm), which is at least interesting.
There’s also Tox (chat/protocol) if you want totally distributed chat.
Personally, I really like distributed models from a theoretical standpoint; but for end-user applications they pose very difficult constraints, we live in a world with ⪅50% publicly routed IP for one, they fundamentally require immense data replication, latency in peer-finding, bandwidth constraints, and ultimately sub-par UX. I thought IPFS with a way to pay nodes to pin content was a really neat idea, but hasn’t caught on, for example. Not to discourage you, if you think it’s workable then have at it, but I think it at least explains the current state of things.
I really wish Matrix had been more successful, but it has some pretty core problems that prevented it from gaining more traction.
It fell into the same trap as XMPP, though perhaps even worse, with a focus more on its protocol and specification than a single unified product vision. The reference server implementation is slow, and using a language not optimal for its purpose, with alternative server implementations left incomplete and unsupported. It took a long time for them to figure out voice and video and for it to work well, and the “user flow” still isn’t at Discord levels.
I’ve rooted for Matrix for a long time, but as a former XMPP evangelist, to me the writing on the wall says it isn’t suited for success either. I’d love to be wrong, but I don’t see a way through.
What do you think the main problems are?
In terms of performance, there’s Rust in the synapse repo already, and both Conduit (Rust) and Dendrite (Go) seem viable. If one of those projects reaches parity with Synapse, do you think that’ll “fix” matrix?
If not, are there other issues core to matrix design? I’m not that familiar with matrix except as an occasional user that follows a few tech rooms, but I’d love to help out if I’m pointed in the right direction. I’m comfortable with Rust and Go (and do Python at my day job), so if backend performance is a bottleneck, I could make help out.
But if the problems are fundamental to how it’s designed or how the project operates, I’d rather work on other things.
I do think the other home server implementations gaining parity (production-ready) with the reference home server would go a long way. I haven’t run a home server but I’ve heard from those that have that it really has a hard time scaling. (Though this serves as impetus to give it a try over spring break)
Which brings me to the caveats of the protocol, I personally don’t think the design is ideal, it’s more described as a distributed message bus, what I’ve read of the spec it’s over engineered, it made good decisions wrt using modern web technologies (JSON, WebRTC), but it didn’t scope itself to the particular task.
That said, I haven’t written a federated protocol, and they have. But if I was going to, I’d really want to look at Discord and see how to copy a lot of that model, but break parts of it out to facilitate federation:
I originally wrote a huge hypothetical design here that I speculated would fare better, but honestly the specifics become less relevant, point is that the shared state of rooms is a real challenge, and one out of scope for just a federated instant messaging system, and I’m no longer certain it’s viable.
I’m personally more interested in P2P protocols than federated, so that’s the stuff I build in my spare time.
So instead of something like Lemmy or Matrix, I’d have something like BitTorrent or Tor, so nodes just add capacity instead of hosting specific content. You could configure your node(s) to pin specific content (e.g. for backups or latency), but your data would also be distributed to other peoples’ computers.
This provides data redundancy, permanency of the service (no centralization whatsoever), and ease of scaling (every client could store and seed data), but comes with complexity. I think it’s workable though.
Matrix is probably something worth looking at, at least from an intellectual standpoint, for you. It uses shared message state and a DAG, plus some fancy perfect forward secrecy (using Signal’s Double Ratchet algorithm), which is at least interesting. There’s also Tox (chat/protocol) if you want totally distributed chat.
Personally, I really like distributed models from a theoretical standpoint; but for end-user applications they pose very difficult constraints, we live in a world with ⪅50% publicly routed IP for one, they fundamentally require immense data replication, latency in peer-finding, bandwidth constraints, and ultimately sub-par UX. I thought IPFS with a way to pay nodes to pin content was a really neat idea, but hasn’t caught on, for example. Not to discourage you, if you think it’s workable then have at it, but I think it at least explains the current state of things.