Kenn White (@[email protected])
mastodon.social
Attached: 3 images Incredible research at BlackHat Asia today by Tong Liu and team from the Institute of Information Engineering, Chinese Academy of Sciences (在iie.ac.cn 的电子邮件经过验证) A dozen+ RCEs on popular LLM framework libraries like LangChain and LlamaIndex - used in lots of chat-assisted apps including GitHub. These guys got a reverse shell in two prompts, and even managed to exploit SetUID for full root on the underlying VM!
  • @cm0002English
    -3•9 months ago

    no, let’s not blame unaffiliated systems engineers for this dumb shit, thanks

    Oh, yea sorry I forgot AI models actually run in a vacuum and needs no supporting code or infrastructure to make it usable to the average user so it doesn’t even need non-AI best security practices! Process isolation? OS hardening? Pfft who needs it

    • flere-imsahoEnglish
      10•9 months ago

      i wouldn’t touch the llm stuff with a barge pole unless i was expressly told to do so, and if i’ve been told to do it, i’d look for another employer (which i’m currently doing, for tangentially-related reasons).

      and it’s not that i don’t care about the llms. i do care very much about them all ending in fiery pit of the deepest of hells.

      • @cm0002English
        1•9 months ago

        Geez, what did an LLM do to you?

        I can see the giant corporation’s proprietary ones, but what’s wrong with open source locally run LLMs to illicit such a response?