does compunction have some meaning I’m not aware of that would apply to statistical prediction machines without human thoughts or emotions such as guilt
Yes we can apply these concepts to models the only thing stopping us is people trying to dabble in philosophy at us.
It’s not a philosophical quibble, but a technical limit with a dangerous social implication. Ascribing human traits its technology doesn’t support is a dangerous deception that cons people into trusting it in ways they really shouldn’t.
To put it bluntly: LLMs don’t understand ethics. They don’t understand anything. That’s not what they’re made for.
They’re highly complex and technically impressive text generators iteratively evaluating a context of preceding tokens to predict, select and append subsequent ones, which then become part of that context for the next iteration. Essentially, they’re very good at guessing what a human-like response might be.
However, that context bias makes them susceptible to following their prompter into what we might consider unsavoury territory. You might implement some filtering mechanisms to curb known topics, but you’d only be playing whack-a-mole with certain symptoms, not addressing the fact that they have no understanding of semantics.
The reason they “hallucinate” is that they don’t even know what’s real. They don’t know the actual concepts these sequences of bytes are supposed to represent. They’re much better than us at correlating language with language, but not with reality. It they don’t know that it’s critical for one sequence of bytes to exactly match some second sequence of bytes to be a valid citation, and that the preceding bytes need to correlate with some bytes associated with that second sequence in a certain way for the pattern of a citation to make sense as a citation.
That’s the reason everything now has warnings about double-checking the output: we all know it’s unreliable. To suggest that it could be reliable in some specific way is disingenuous and irresponsible.
To pretend that it can be held to a human standard of morality is outright bullshit.
Are you saying LLMs can NEVER be safe, because you can always bury their safety rails in context?
In which case, the sad result will probably be to absolve AI companies of liability of things they can’t control, because the other option would be to legislate chatbots out of existence and then how do we get to the utopia of a workerless society? /s
Are you saying LLMs can NEVER be safe, because you can always bury their safety rails in context?
For a serious answer?
It’s not like humans follow safety instructions all the time either. A machine built to imitate humans will be prone to mistakes too. Any safety from within a system are liable to fail if the system itself makes a mistake.
LLMs have additional limitations compared to humans, but even if we developed and attached some technical model of semantics and morality, the resulting models would at best be an approximation.
For tasks where that safety or accuracy doesn’t matter as much, that’s fine. Pen-testing a system or hunting bugs doesn’t need to strictly abide by certain rules (and in fact may profit from being unconstrained by the assumptions of engineers). Scanning a body of text for irregular patterns or phrases correlating with some keywords where you don’t know the exact words could be useful in covering a lot more than a human might immediately think off.
It becomes a problem where accuracy does matter. Scientific work, legal matters, anything involving human safety, sensitive data for example. Here, the risk is that they can make a lot of mistakes much faster. Particularly with large-scale tasks, thoroughly checking the results can be tedious, exhausting and tempting to just shortcut and trust the machine (and you know how lazy people tend to get with tedious, exhausting tasks, so it would be irresponsible to assume they’d be on the ball here).
This is where safety instructions can become dangerous to rely on, because the same unreliability that produces mistakes may also discard the safety instructions.
Letting Claude manage your production environment with its “safety rails” only coming in the form of instructions what not to do is a timebomb for getting your production systems messed up.
Effective safeguards have to come from outside whatever they’re supposed to constrain. Supervisors that are supposed to monitor their workers’ adherence to protocols, for instance, or mechanisms that physically make mistakes harder (like locking out, tag out).
No safeguards are absolute, and as the saying goes, whenever you invent a foolproof system, the universe invents a greater fool. Stuff like putting a zip-tie around dead man switches on tools or machines that are supposed to shut off the machine if the operator loses control for whatever reason, because it’s inconvenient having to keep holding the button and much easier to tell yourself “it’ll be fine” (until it isn’t).
In some cases, it can also be really difficult to handle certain dangers. But it’s worth trying at least, and that requires first acknowledging those problems. In the case or LLMs, that means acknowledging their limits and pitfalls, transparently and honestly communicating them and-
the sad result will probably be to absolve AI companies of liability of things they can’t control
That… wasn’t my plan, but I’m too poor for my plans to matter anyway. Valid point.
the other option would be to legislate chatbots out of existence
Well, maybe with education and transparency, we could…
Who am I kidding.
how do we get to the utopia of a workerless society?
I propose an experiment where everyone working for the people peddling that dream stops doing so. I haven’t worked out the details yet, but I think they should get the chance to experience that utopia. ;-)
You’re taking too broad a scope to my question. Let me parse it down: to a situation:
Can you EVER create a fully-functioning (ie non-gimped) LLM that will NEVER encourage a user to kill themselves? Because my understanding of the technology is that the longer you talk to an LLM, the further it gets from its training and guardrails, the more weirdness can happen. People want a free-wheeling chatbot that can adapt to their lives and personality and data, so, short of resetting their context window regularly, I don’t know how to keep guard rails in place through hundreds of hours of conversation.
Some of what you say sounds familiar, then quis custodiet ipsos custodes?–who watches the watchmen–bubbled up in my mind. It is a question that has long plagued humanity.
Right, let’s not talk about the use of language. Let’s talk about emotional capacity as a strictly biological phenomenon, emerging from the interactions of various neurotransmitters and constructs of cells.
What makes you think a language model can feel guilt?
Uhm a feeling to a model can just be a higher or lower value of a control node or something developed internally it isn’t special or anything just an existing concept that applies. I think you are assigning superficial meaning to words as if they are what makes us special, if that were true LLM’s would win ha.
Yes we can apply these concepts to models the only thing stopping us is people trying to dabble in philosophy at us.
It’s not a philosophical quibble, but a technical limit with a dangerous social implication. Ascribing human traits its technology doesn’t support is a dangerous deception that cons people into trusting it in ways they really shouldn’t.
To put it bluntly: LLMs don’t understand ethics. They don’t understand anything. That’s not what they’re made for.
They’re highly complex and technically impressive text generators iteratively evaluating a context of preceding tokens to predict, select and append subsequent ones, which then become part of that context for the next iteration. Essentially, they’re very good at guessing what a human-like response might be.
However, that context bias makes them susceptible to following their prompter into what we might consider unsavoury territory. You might implement some filtering mechanisms to curb known topics, but you’d only be playing whack-a-mole with certain symptoms, not addressing the fact that they have no understanding of semantics.
The reason they “hallucinate” is that they don’t even know what’s real. They don’t know the actual concepts these sequences of bytes are supposed to represent. They’re much better than us at correlating language with language, but not with reality. It they don’t know that it’s critical for one sequence of bytes to exactly match some second sequence of bytes to be a valid citation, and that the preceding bytes need to correlate with some bytes associated with that second sequence in a certain way for the pattern of a citation to make sense as a citation.
That’s the reason everything now has warnings about double-checking the output: we all know it’s unreliable. To suggest that it could be reliable in some specific way is disingenuous and irresponsible.
To pretend that it can be held to a human standard of morality is outright bullshit.
Are you saying LLMs can NEVER be safe, because you can always bury their safety rails in context?
In which case, the sad result will probably be to absolve AI companies of liability of things they can’t control, because the other option would be to legislate chatbots out of existence and then how do we get to the utopia of a workerless society? /s
For a serious answer?
It’s not like humans follow safety instructions all the time either. A machine built to imitate humans will be prone to mistakes too. Any safety from within a system are liable to fail if the system itself makes a mistake.
LLMs have additional limitations compared to humans, but even if we developed and attached some technical model of semantics and morality, the resulting models would at best be an approximation.
For tasks where that safety or accuracy doesn’t matter as much, that’s fine. Pen-testing a system or hunting bugs doesn’t need to strictly abide by certain rules (and in fact may profit from being unconstrained by the assumptions of engineers). Scanning a body of text for irregular patterns or phrases correlating with some keywords where you don’t know the exact words could be useful in covering a lot more than a human might immediately think off.
It becomes a problem where accuracy does matter. Scientific work, legal matters, anything involving human safety, sensitive data for example. Here, the risk is that they can make a lot of mistakes much faster. Particularly with large-scale tasks, thoroughly checking the results can be tedious, exhausting and tempting to just shortcut and trust the machine (and you know how lazy people tend to get with tedious, exhausting tasks, so it would be irresponsible to assume they’d be on the ball here).
This is where safety instructions can become dangerous to rely on, because the same unreliability that produces mistakes may also discard the safety instructions.
Letting Claude manage your production environment with its “safety rails” only coming in the form of instructions what not to do is a timebomb for getting your production systems messed up.
Effective safeguards have to come from outside whatever they’re supposed to constrain. Supervisors that are supposed to monitor their workers’ adherence to protocols, for instance, or mechanisms that physically make mistakes harder (like locking out, tag out).
No safeguards are absolute, and as the saying goes, whenever you invent a foolproof system, the universe invents a greater fool. Stuff like putting a zip-tie around dead man switches on tools or machines that are supposed to shut off the machine if the operator loses control for whatever reason, because it’s inconvenient having to keep holding the button and much easier to tell yourself “it’ll be fine” (until it isn’t).
In some cases, it can also be really difficult to handle certain dangers. But it’s worth trying at least, and that requires first acknowledging those problems. In the case or LLMs, that means acknowledging their limits and pitfalls, transparently and honestly communicating them and-
That… wasn’t my plan, but I’m too poor for my plans to matter anyway. Valid point.
Well, maybe with education and transparency, we could…
Who am I kidding.
I propose an experiment where everyone working for the people peddling that dream stops doing so. I haven’t worked out the details yet, but I think they should get the chance to experience that utopia. ;-)
You’re taking too broad a scope to my question. Let me parse it down: to a situation:
Can you EVER create a fully-functioning (ie non-gimped) LLM that will NEVER encourage a user to kill themselves? Because my understanding of the technology is that the longer you talk to an LLM, the further it gets from its training and guardrails, the more weirdness can happen. People want a free-wheeling chatbot that can adapt to their lives and personality and data, so, short of resetting their context window regularly, I don’t know how to keep guard rails in place through hundreds of hours of conversation.
Some of what you say sounds familiar, then quis custodiet ipsos custodes?–who watches the watchmen–bubbled up in my mind. It is a question that has long plagued humanity.
Uhg I am not interested in reading your philosophy I really don’t care
Then what do you call ascribing human sentiment to a weighted random text generator?
Ascribing human sentiment to a weighted random text generator.
Then why is describing the technical limitations of a weighted random text generator philosophy?
Because discussing language and how it is used can be philosophy… This is really painful dude but I am here for it.
Right, let’s not talk about the use of language. Let’s talk about emotional capacity as a strictly biological phenomenon, emerging from the interactions of various neurotransmitters and constructs of cells.
What makes you think a language model can feel guilt?
Uhm a feeling to a model can just be a higher or lower value of a control node or something developed internally it isn’t special or anything just an existing concept that applies. I think you are assigning superficial meaning to words as if they are what makes us special, if that were true LLM’s would win ha.