Hello all,
Wanted to open a discussion on Lemmy’s post sorting options right now. I don’t have any experience with implementing this type of thing but right now the algorithm appears… Off? For example, ‘Active’ gives me a lot of posts over a day old but ‘Hot’ may as well be ‘New’ i.e. more recent posts with little engagement.
I don’t know if it’s due to Lemmy still picking up steam or a fundamental flaw with the algorithm. Like I said, I’m really curious to hear the opinions of those more knowledgeable.
I don’t know a lot about Lemmy’s implementation but a difficult thing to deal with is how do you “rank” a post? Like you have a small community of a few active people, but there’s federation with a massive community with lots of users - which posts are “better”?
Worse still, there’s an inherent lag/delay with the federated posts, a post that was very active in the last hour might have only been federated to the server in the last 5mins - so what do you do, do you bubble up all those posts or ignore it because there’s more recent and relevant things?
The kicker is that these decision points aren’t instant either, any system that’s doing this kind of ranking will have an algorithm as you describe, but that algorithm will take time to process all the data, while the data is coming in batches as each server federates with each other. It’s a difficult problem to solve.
It’s all open source, you can look at the algorithms here. https://join-lemmy.org/docs/en/index.html https://join-lemmy.org/docs/en/contributors/07-ranking-algo.html
I’m aware, what I am getting at is that there’s multiple “Right” answers to solving what is essentially a very difficult problem.
I see this as a really clear win over the likes of Reddit and Facebook of making the algorithm more understandable to users so they see WHY they’re being fed the information they’re getting.
Also, it has to be designed in a way that limits federated servers from gaming the system.
I think they should implement a “best” category and allow each server to curate which communities it adds to it’s “best” category and just let them game it all they want. Just dont replace any of the current options with it. Then if you select “best” and “all” it would give you content that every non black listed server from your home things is best.
I think this is where federation will get the most interesting. It would be cool to add/remove weight to some instances and communities. And each instance’s software could handle this differently with different algorithms.
I’m completely speculating because I’m high. I have no idea what Lemmy’s roadmap looks like. Would be cool though man.