The AI Horde is a project I started in order to provide access to Generative AI to everyone in the world, regardless of wealth and resources. The objective is to provide a truly open REST API that anyone is free integrate with for their own software and games and allows people to experiment without requiring online payment that is not always possible for everyone.

It is fully FOSS and relies on people volunteering their idle compute from their PCs. In exchange, you receive more priority for your own generations. We already have close to 100 workers, providing generations from stable diffusion to 70b LLM models!

Also the lemmy community is at [email protected]

If you are interested in democratizing access to Generative AI, consider joining us!

  • db0OPEnglish
    2•1 year ago

    Petals does distributed inference, which even with their enhanced system, is still pretty slow. The AI Horde sends the whole prompt to one individual worker and expects the reply back from that one worker. As such the speed is only dependent on the GPU that worker has. This limits us to how big models we can run, but then again I’ve seen people onboarding some crazy GPUs on the horde.

    • @BlaedMEnglish
      2•1 year ago

      Appreciate the insight. I like that approach. I just learned you can become an Alchemist too. That’s a nice touch.

      Alternatively you can become an Alchemist worker which is much more lightweight and can even run on CPU (i.e. without a GPU).

      If you’re reading this, consider turning your PC into a Horde Worker! It’s a great way to contribute to AI (text & image-based) if you have the power to spare.