NVIDIA just released a free new chatbot that runs locally on Windows PCs. You need a GeForce RTX 30 Series GPU or higher and at least 8GB of VRAM to use the bot.

• NVIDIA released a demo version of a chatbot that runs locally on your PC, giving it access to your files and documents.

• The chatbot, called Chat with RTX, can answer queries and create summaries based on personal data fed into it.

• It supports various file formats and can integrate YouTube videos for contextual queries, making it useful for data research and analysis.

    • @KyrgizionEnglish
      4•8 months ago

      2xxx too. It’s only available for 3xxx and up.

    • @anlumoEnglish
      2•7 months ago

      The whole point of the project was to use the Tensor cores. There are a ton of other implementations for regular GPU acceleration.

    • @CeeBeeEnglish
      2•7 months ago

      Just use Ollama with Ollama WebUI

      • DojanEnglish
        19•8 months ago

        There were CUDA cores before RTX. I can run LLMs on my CPU just fine.

      • halfwaythereEnglish
        6•8 months ago

        This statement is so wrong. I have Ollama with llama2 dataset running decently on a 970 card. Is it super fast? No. Is it usable? Yes absolutely.