This is still very much early-days, but I’m thinking about building a distributed link aggregator. Some notes:
- no servers (all P2P) except to connect peers
- no “instances” so everything is in one namespace (so like reddit /r/community, not lemmy instance/c/community)
- everyone is a moderator - moderation based on a web of trust type system
- desktop first, mobile/web later - web would require servers, mobile may have platform-specific issues
- no plans to integrate with fediverse, but could be possible
- initial intent is to be text-only - file storage could be supported later in an opt-in basis (for now, just link to an external img host), but I’m worried about disk storage since there’s no central data lake and everyone needs to chip in some storage space
Some technical details:
- written in Rust and ReactJS - Tauri-based
- uses Iroh for data synchronization
- looking into Appleseed and maybe TrustNet for moderation
Current status:
- rewrote Appleseed and part of TrustNet to Rust for perf testing last weekend (Rust is waaay faster) - will probably rewrite again once I finish reading the paper (current code is a direct port)
- have basic Tauri+Iroh app, but it’s a glorified chat app w/ no syncing between peers (a peer can join a chat, but will lose access if the host goes down)
- reading source code for SimpleX (re-learning Haskell too) - could be interesting for DMs or something
I’m not sure how long I’ll work on this, but I’ll probably release some libraries under a relatively liberal license, so something like LGPL at the most stringent, but probably Apache/MIT (depends on if I need to clean-room the appleseed implementation).
So, if there’s interest here, I’m happy to post updates for discussion periodically.
You must log in or register to comment.
Idk sounds like you should just work on Lemmy / the fediverse & not require peers to be servers (often online). It’s not really a good idea, you don’t gain much from being fully p2p, and you will fuck up the user experience a lot
I have submitted some patches here and there to Lemmy when something bothers me, this is just an alternative in case it takes off (unlikely) and has issues scaling. There’s a ton of duplication of data, and that adds up, and that’s hard to square without a profit model.
So that’s the main problem I’m trying to solve here, and I think it’s fundamental to how the fediverse is designed. That, and it’s annoying having to care about instances, which also is a UX issue. I think I can mitigate a lot of the UX issues by hosting a few mirrors to improve availability, and hopefully the community steps up to spread out that cost. It should be a lot cheaper than Lemmy because there’s less duplication, and it should be temporary (hobbyists will hopefully run large nodes, kind of like how Tor operates).
But you’re right, it’s a pretty big risk because it’s a lot of work for something that might end up being worse. It’s an attractive option with internet speeds getting better, but I know IPFS is kind of a crappy experience and Iroh is unstable and unproven, so it’s a gamble. The theoretical underpinnings look good (better DHT, less bloat, etc), and maybe pieces could be reused in something like Lemmy, like the moderation engine, so I think it’s worth the effort to play with.
Thanks for the feedback, if you have specific criticisms and potential solutions of p2p apps, I’m interested. I’ve been interested in p2p for years, and this seemed like a good excuse to play around with it.
Thanks - yeah, I think any app that’s not capable of being a simple website is DoA. I’ve seen many attempts at building something in this style, and none of them have really taken off. Social media like diaspora and marketplaces like lighthouse. The closest is BitTorrent. I don’t consider instances to be any more confusing than email domains. Having your phone running a background service would definitely be the worst outcome
If the final result is that peers need to be online for their data to be accessed, and it’s not replicated around much, then it’s no different than server-client. IPFS is definitely a tragedy when it comes to trying to use it, and it has blacklisting built in. Reality: there is a small cartel of hosts formed, and the effort to run your own instance is way too high when you could just use cloudflare’s. I consider IPFS to be a fully failed experiment that didn’t even take off in crypto
Again, my suggestion is that you take your passion and obvious brilliance, and apply it to greatly improving products that already have traction. If you have ways to make moderation better and network efficiency higher, then you’d be directly impacting the lives of tens of thousands of Lemmy users, or whatever software you chose to aid. One of these cases where practicality beats ideal design. Unless you really just want to experiment and learn some stuff for your own sake
simple website
WebRTC might be sufficient, just like it is for libp2p.
My app is all in React using Tauri, so I would just need to replace a few calls to a local library with webRTC support, which should be pretty easy.
phone running a background service
That would be 100% opt-in. I would even allow disabling sync over data, so you could still post and whatnot offline, and it would sync with peers later.
But that’s also why I’m doing desktop first, I don’t need you worry about limitations while I’m prototyping it.
peers need to be online for their data to be accessed
I intend to have enough duplication that it shouldn’t be an issue.
To start, I also intend to have always online “clients” that, combined, have “all” (as much as any p2p app can) of the data until enough people use it to have it self-host.
apply it to greatly improving products that already have traction
That’s usually what I do anyway. I really don’t want to head a project, but, as you mentioned, other stuff in this space seems to have failed:
- IPFS - too slow, Iroh improves that
- Hyphanet (OG freenet) - focused on “web pages” and anonymity, which attracted the wrong people
- new freenet - isn’t available yet (soon!!) - once it is, I’ll play with it, but it’s libp2p-based, so probably slow
- scuttlebutt - only small groups, I’m looking for a Reddit alt
- aether - ephemeral in nature (6 months), and the project seems dead (no activity for 3-4 years)
- cryptocurrency-related - almost everything else, which it attracts the wrong crowd
I’ve contributed a bit to Lemmy, but I think lemmy is fundamentally flawed so we’ll need an alternative eventually. I hope to make a bridge to act like an instance eventually, which would at least allow a federation of p2p networks (could be cool).
ways to make moderation better
Better is subjective, but yeah, I’ll definitely be pulling from Lemmy datasets for simulations. However, it would probably need to be a browser extension due to the data and interaction I want.
There are three types of trust:
- trusted - I have proactively approved something (most web of trust systems)
- untrusted - no productivity whatsoever
- distrusted - I have proactively flagged something as harmful
Lemmy mods handle the third and user votes kind of handle the first two, but if you don’t trust the mods, you’re SOL. It’s also at risk of Sybil attacks (e.g. vote brigading), so it’ll have similar issues as Reddit.
I think I can all three types of trust with two inputs: flags and votes. Explicit trust of a user could be helpful (e.g. bots or community mods), but hopefully not required. If it significantly improves performance, I might add a “mod queue” so users can train the moderation system (e.g. do you agree with this moderation decision?).
I think it’s an interesting area to explore, and I don’t see anything using it, which means it either:
- doesn’t work - so I shouldn’t propose it to lemmy
- is hard - so experimentation is valuable
- is too expensive - probably the case (initial naïve simulation with 100k nodes took ~3s to initialize, checking is fast)
I’m hoping people think it’s harder than it is, which is why it’s overlooked. Votes and blessed moderation are easy to implement, so I’m banking on that. There may also be a way to integrate some limited machine learning and get hardware acceleration from a GPU or AI chip (e.g. Google Pixel’s AI chip), so “expensive” may be less of an issue once I go public.
Anyway, thanks for the thoughts! I may need to discuss web support with Iroh devs. I know they have a web demo, but I think it routes through a web server, not an actual p2p web app, but I haven’t checked.
Cheers bro, great to meet you and I’m sure we would get along. Hoping for your success and joy
You as well. I hope you have an excellent day. :)
Reminds me a lot of freenetproject
Yup, that’s kind of the idea, but less “free speech with strong anonymity” and more “user-driven moderation.”
I might end up building something on top of the new freenet, but I haven’t looked into it too much. The old one (now hyphanet) isn’t really what I’m looking for.
Basically, I want something like lemmy, but:
- no mods - but still have a moderation solution
- one namespace - so /c/community, not intanceA/c/community and instanceB/c/community
- minimal duplication - enough for availability, but much less than lemmy
This sounds very interesting. I am also working on a p2p based social network that tries to bring the idea and framework of social norms into a digitized format and it seems very inline with what you are trying to achieve. I already have a prototype up and running so it would be cool if we talked more about our ideas.
DM me if you want to talk more.
Yeah, I’m interested, but I’m not sure how much I’ll commit to it. If I’m still interested in this project in a couple weeks once I’ve solved the fun problems, I’ll probably be interested in helping out with another project.
I hope to post some source code soon, at which point I think I’d be ready to discuss other projects and whatnot.
If you post something, I’ll definitely look at it. Otherwise, I’ll try to reach out in a week or two once I’ve gotten my current project out of my head.
This sounds great! I really dislike that on lemmy, I can wake up any day and find out the instance I have my community on is gone, and I lose all my posts and subscribers. I’ve been thinking some sort of a p2p solution would be great, but I had no idea if it would actually be possible. I look forward to seeing how this progressed. I’d love to be able to see updates. Thanks for sharing!
Yeah, that’s the dream with p2p apps. But whether it works like that in practice is another thing entirely.
So yeah, I’ll post occasionally then.
