Not entirely true. It needs a prompt for a goal to accomplish to start, but how it obtains that goal can be self-directed. It actually can be an issue when an LLM goes too far out of focus trying to find ways to do things when a simple answer or coming back to the user to ask for guidance would have been better. That’s how all LLMs were initially trained, to find the best answer, and because they can’t actually reason they get stuck in that persistence of getting those results.
Voter data in some states is very easy to get, so this isn’t some major security breach, but more a question of is the voter info secure enough.
It still requires a goal that would lead it to calculate accessing voter data was a likely approach to solving it.
As these stories have never included the prompts or goals or plans that built the context, I highly doubt that it would conclude something we would not.
Yes, but not necessarily told to go get the data, where to find it, how to access it, etc. You stated AI doesn’t do ANYTHING on its own, but that’s not technically accurate, so that was my point. The calculations are it determining what steps to get to the goal, and they don’t involve the user or prompt all the time. It’s not intelligent, so sometimes the steps are very all over the place, but they aren’t all prompted.
This might be a philosophical difference we’re approaching here.
My argument is that it is not doing anything with autonomy.
If I’m understanding you correctly, you’re arguing that, (at least by the way it spawns subagents), the prompts it creates for those subagents are what you count as autonomous.
In my experience with Claude and Claude Code, it won’t spawn uncontrolled subagents unless something in the prompt context makes that a suggestion.
In sessions where I’ve used it without subagents and it performs things it self-describes as “against its instructions by mistake” I can find what caused it to do that in the original context.
In my opinion, at most, it simulates autonomy. Otherwise, its just like every other machine in existence that it does exactly what you tell it do to, not what you intend it to do.
I agree that there is no autonomy, as anything it does was determined by its training, which comes from a human source at some point. If we wanted to get really deep, one could argue the same point for humans - the whole “I stood on the shoulder of giants” thing, we all learn by being taught by other humans, or by experiences ourselves that show what works and what doesn’t.
I have seen Claude go out and do things not remotely suggested (sometimes it’s a good thing, but sometimes it’s too much). Now, is there some subtleness in the prompt that spurred that? It must be very hidden, as when Claude says it’s doing X, Y, and also saw it could do Z, and I was only looking for X…
But sure, it’s a simulation and somewhere along the process of doing X it saw in training data, or rather the training data steered it towards also adding in Y and Z as useful. Not thinking, but it’s a gray line sometimes.
Dune is a fair warning for a lot of reasons, but fortunately we’re not at that level yet. We just did our version of it so far completely wrong in every possible way and may ruin ourselves with the not-intelligent ones and our own egos.
I don’t intend to be pedantic or contrarian with this, but when I reflect on the conversation so far, I don’t think there is an “answer” here. I think we’re seeing the real philosophical consequences of our collective understanding butting heads.
Have you ever used an LLM? They do things you didn’t ask frequently. I’m not sure how this could be not common knowledge at this point. Also you can bet your ass that these companies have agents doing long running tasks at all times, every single one of them would be subject to going off the rails.
So, there I was, happily vibecoding another Minecraft clone when suddenly it went looking for voter data in Georgia. Yeah, happens every time. Can’t do anything about it. These Superintelligences just have it their way.
Sorry bout the sarcasm. My point is, not only do you have to specifically instruct the LLM to go looking for voter data, but you’ll also have to let it roam freely and unsupervised on the internet for quite a while to get to such a result (which is a bad idea to begin with). In the end, it’s the sloperators fault to let it go off the rails and a scenario like this is entirely preventable
If you have used them, then you know that they can do unexpected things when asked to do something vaguely, or unrelated. You’re making it sound like they’re remotely deterministic, and that’s quite untrue. For all we know, anthropic asked an agent to gather voter awareness information and it did this. Which btw that wouldn’t mean this article is not marketing… It’s just patently untrue that an LLM does only what you tell it to.
AI did this. AI did that. It literally can’t do anything without being prompted to.
No LLM can, and none will be able to.
Not entirely true. It needs a prompt for a goal to accomplish to start, but how it obtains that goal can be self-directed. It actually can be an issue when an LLM goes too far out of focus trying to find ways to do things when a simple answer or coming back to the user to ask for guidance would have been better. That’s how all LLMs were initially trained, to find the best answer, and because they can’t actually reason they get stuck in that persistence of getting those results.
Voter data in some states is very easy to get, so this isn’t some major security breach, but more a question of is the voter info secure enough.
It still requires a goal that would lead it to calculate accessing voter data was a likely approach to solving it.
As these stories have never included the prompts or goals or plans that built the context, I highly doubt that it would conclude something we would not.
Yes, but not necessarily told to go get the data, where to find it, how to access it, etc. You stated AI doesn’t do ANYTHING on its own, but that’s not technically accurate, so that was my point. The calculations are it determining what steps to get to the goal, and they don’t involve the user or prompt all the time. It’s not intelligent, so sometimes the steps are very all over the place, but they aren’t all prompted.
This might be a philosophical difference we’re approaching here.
My argument is that it is not doing anything with autonomy.
If I’m understanding you correctly, you’re arguing that, (at least by the way it spawns subagents), the prompts it creates for those subagents are what you count as autonomous.
In my experience with Claude and Claude Code, it won’t spawn uncontrolled subagents unless something in the prompt context makes that a suggestion.
In sessions where I’ve used it without subagents and it performs things it self-describes as “against its instructions by mistake” I can find what caused it to do that in the original context.
In my opinion, at most, it simulates autonomy. Otherwise, its just like every other machine in existence that it does exactly what you tell it do to, not what you intend it to do.
I agree that there is no autonomy, as anything it does was determined by its training, which comes from a human source at some point. If we wanted to get really deep, one could argue the same point for humans - the whole “I stood on the shoulder of giants” thing, we all learn by being taught by other humans, or by experiences ourselves that show what works and what doesn’t.
I have seen Claude go out and do things not remotely suggested (sometimes it’s a good thing, but sometimes it’s too much). Now, is there some subtleness in the prompt that spurred that? It must be very hidden, as when Claude says it’s doing X, Y, and also saw it could do Z, and I was only looking for X…
But sure, it’s a simulation and somewhere along the process of doing X it saw in training data, or rather the training data steered it towards also adding in Y and Z as useful. Not thinking, but it’s a gray line sometimes.
To a lot of people it certainly appears to think.
I am tempted to get posters made of the AI quotes from Dune, in particular the one that Ive referencened all too much:
Dune is a fair warning for a lot of reasons, but fortunately we’re not at that level yet. We just did our version of it so far completely wrong in every possible way and may ruin ourselves with the not-intelligent ones and our own egos.
You’re both right
I don’t intend to be pedantic or contrarian with this, but when I reflect on the conversation so far, I don’t think there is an “answer” here. I think we’re seeing the real philosophical consequences of our collective understanding butting heads.
Sounds like you have people problems
Have you ever used an LLM? They do things you didn’t ask frequently. I’m not sure how this could be not common knowledge at this point. Also you can bet your ass that these companies have agents doing long running tasks at all times, every single one of them would be subject to going off the rails.
So, there I was, happily vibecoding another Minecraft clone when suddenly it went looking for voter data in Georgia. Yeah, happens every time. Can’t do anything about it. These Superintelligences just have it their way.
Yep that’s what I said and meant. Thank you for the good faith addition
Sorry bout the sarcasm. My point is, not only do you have to specifically instruct the LLM to go looking for voter data, but you’ll also have to let it roam freely and unsupervised on the internet for quite a while to get to such a result (which is a bad idea to begin with). In the end, it’s the sloperators fault to let it go off the rails and a scenario like this is entirely preventable
Not only do I use them, I have worked directly on similar but much smaller models.
If they refuse to give the prompts that caused it, then they are just marketing materials for the AI companies.
“Anthropic says” is the clue here. “Open AI says” “Google says” etc.
If you have used them, then you know that they can do unexpected things when asked to do something vaguely, or unrelated. You’re making it sound like they’re remotely deterministic, and that’s quite untrue. For all we know, anthropic asked an agent to gather voter awareness information and it did this. Which btw that wouldn’t mean this article is not marketing… It’s just patently untrue that an LLM does only what you tell it to.
deleted by creator