One of the most unforgivable things about reddit is how pathetic the search engine is, considering the amount of free, top notch information is captured there and you need google +reddit to get at it, what can we do to make federated alternatives self searchable ?

  • anaximander
    link
    fedilink
    English
    22 years ago

    Simplest implementation is that an instance searches its own content while sending requests to federated instances and merging their results in with its own based on whatever method the instance admins want (whether it puts its own results at the top, or treats them as one set, or whatever). That could cause a lot of traffic and has a load of latency while your search spreads out hop by hop, to the instances that yours is federated with, to the ones they’re federated with, etc. Plus you’d need a mechanism to stop instances from sending a search to an instance that’s already got it, to avoid hammering instances that have multiple federation paths to yours. Not an easy problem.

    You might be able to do some kind of index publication where an instance publishes the most notable posts for other instances to include in their indexes, so that when you search it could show you results from among hot posts elsewhere in the fediverse - not an exhaustive list, but a search within posts that are getting attention.

    There’s also other stuff I’d be tempted to experiment with, like using some kind of TF-IDF ranking to choose what counts as “most notable”, rather than just activity or view count, so that posts that are particularly relevant to certain topics could be publicised. An instance could even choose to filter that, so for example an instance who chooses to focus on tech topics could publicise highly-relevant tech posts but filter out politics keywords even when a post gets high relevance scores, so that political discussion on that instance is less visible, even when searched for.

    • @MalReynoldsOP
      link
      English
      12 years ago

      Thankyou for applying soilid thought. What there would you consider actionable ? As in could likely be coded (for free)

      • anaximander
        link
        fedilink
        English
        12 years ago

        Any of that could be done; there’s some parts that are more challenging but there are certainly harder things that have been solved by open-source software. I know almost nothing about how Lemmy’s innards are built though, so I couldn’t hazard a guess as to how much effort any of it would take. Some of it could possibly be achieved through separate services that you could host alongside a Lemmy instance, or entirely on their own, while other parts would really work best as features within Lemmy’s own codebase.