Hi all,

Decided I want to give Hermes a try, but man their documentation is horrid. So, for running fully offline, here’s their “guide”: https://hermes-agent.ai/how-to/run-hermes-offline

Now since that’s trash, anyone here actually doing this? Did you just remove all references to non-local providers in the config.yaml? Are you using other containers for TTS and STT? Any help would be appreciated as I dive in.

Thanks in advance.

  • pechEnglish
    1·
    22 hours ago

    The info might be a little scattered throughout the various topics, but they have a suggestion on their “setting the main model” page:

    providers:
  local-vllm:
    api: http://127.0.0.1:8000/v1
    models:
      my-reasoner:
        answer_in_reasoning: true

    This goes in the config.yaml, it used to be ‘base_url’ instead of API, but the structure should be close enough reason through.

    • FmstratOPEnglish
      1·
      18 hours ago

      Yea, connecting to my local llama instance was pretty straight forward. But then I realized there are still moe providers set up with openai and anthropic outside of the main providers, and then there are MCP tools and skills that don’t ask permission to use the internet, etc. Just seems like OpenWebUI gives you a lot more visibility and control.

      • pechEnglish
        2·
        13 hours ago

        Yeah I kind of got tired trying to manage all the changes the devs made to the harness, though the docs are explicit in saying nixos is a tier 2 platform lol.

        I’ve been using deepseek-harness for a few weeks and it’s been awesome