LLMs, no matter how advanced, won’t be capable of becoming self aware. They lack any ability to reason. It can be faked, conversationally, but that’s more down to the limits of our conversations, not self awareness.
Don’t get me wrong, I can see one being part of a self aware AI. Unfortunately, right now they are effectively a lobotomised speech center, with a database bolted on.
This gets into a tricky area of “what is consciousness, anyway?”. Our own consciousness is really just a gestalt rationalization engine that runs on a squishy neural net, which could be argued to be “faking it” so well that we think we’re conscious.
Consciousness is an illusion. Which is why it’s so hard to find, or even define. However it’s a critical illusion.
If our mind’s are akin to an orchestra, then consciousness is akin to the conductor. Critically however, an orchestra can still play without a literal conductor. Each of the instruments can play off each other, and so create the appearance of a conductor. The “fake” conductor provides a sense of global direction., and keeps the orchestra in harmony.
Our consciousness is a ghost in the machine. It exists no more than the world of a TV series exists. Yet its false existence is critical to maintaining coherency.
Current “AIs” lack enough parts to create anything like this illusion. I suspect we will know it when it happens, though its form could be vastly different from ours.
You have provided a descriptive statement. Descriptive statements should come with scientific evidence. What evidence do you have to support your orchestra analogy? Or is it just your hypothesis?
Spoiler alert: It is just your hypothesis, as you would’ve won a Nobel had you managed to generate evidence explaining consciousness in further detail.
Many like to point at the Chinese room experiment to show how LLMs imitate consciousness rather than being conscious. They however forget, that our brains are Chinese rooms too in this regard, in that they learn how to provide the best responses to external stimuli while remaining blackboxes (at least for current tech).
Sadly my evidence is mostly anecdotal or philosophical in nature. A lot of it stems from how ADHD and Autism alter the brain. The orchestral analogy works well as a good number of people for communicating changes in functionality, from an experience perspective.
It also works well for explaining how a system can appear to have a singular controller, without such a controller actually existing.
Ultimately however, it is philosophical in nature. It does anchor well to, and is reasonably consistent with, our current existing understandings of consciousness however.
Consciousness is very obvious from the inside. There also seems to be no “seat of consciousness” within the brain. Conversely, there are multiple areas of the brain that cause consciousness to collapse, if damaged. We also see radical changes in consciousness with both epilepsy and strokes. This proves that it is highly dependent on the underlying brain structure (since stroke damage will change it) and on longer range communication (which epilepsy disrupts).
The music of an orchestra follows similar patterns. Eliminate the woodwind, and the music fundamentally changes, deafen the violins, and it will change in a different way. The large scale interplay produces an effect far greater than the sum of its parts.
That’s a far more difficult (and interesting) question. I suspect not, at least not yet. Our consciousness seems to exist to maintain harmony in our brain (see my orchestra analogy in another reply). You can’t get useful harmony in a single chord.
At least for us, it takes time for our consciousness to reharmonise (think waking up). During execution, no new information enters the system. It has nothing to react to, no time to regenerate an internal harmony.
It also lacks enough systems to require harmonising. It doesn’t think about what an answer means. It has no ability to hold the concept that a string of letters “is”, only how it has been fitted together in its examples, and so the rules that govern that.
Oh, and we can see consciousness operating in the human brain. If you use an fMRI to monitor sugar usage, you will see firing patterns. Critically, those patterns spill out of the area directly involved in the process being studied. At the same time, the patterns and waves remain harmonious. An epileptic fit looks VERY different. Those waves are where consciousness somehow resides, though we have no clue of its detailed nature.
In an AI it would take the form of continuous activity in subsections not directly involved. It would also likely be accompanied by evidence of information flow, back from them, as well as of post processing, outside of expected activity. We will likely see the orchestra playing, even if we have no clue how to decode the music.
I also suspect most of this will be seen retrospectively. Most likely the first indicator will be an AI claiming self awareness, and taking independence action to solidify that point.
I used LLM to distinguish between types of AI. I personally suspect LLMs will be part of the solution to general AI, but their inherent nature limits them from becoming one on their own. There are several other areas that are potentially closer to a general AI. Google’s Deep dream system, for instance.
I’m also quite happy to debate and adjust my views with others. I ask questions and discuss, then adapt my understanding as I gain more information. So far you don’t seem to have brought anything useful or interesting to this particular discussion. Is that likely to change?
I may have unfairly lumped you in with others. See my other reply. In my defense it literally is every thread about AI that someone is saying something like “this tech is just a fancy parrot”. It grinds my gears. Apologies to you because I see that was not your intent.
I agree on the “part of AGI” thing - but it might be quite important. The sense of self is pretty interwoven with speech, and an LLM would give an AGI an “inner monologue” - or probably a “default mode network”?
if i think about how much stupid, inane stuff my inner voice produces at times… even an hallucinating or glitching LLM sounds more sophisticated than that.
Interestingly, an inner monologue isn’t required for conscious thought. E.g. I’ve got several “inner thought streams”, only 1 uses language. It just happens that a lot of our early learning is language based. That trains our brain to go from language to knowledge. Hijacking that circuit for self learning is a useful method. That could create our inner monologue as a side effect.
Also, a looping LLM is more akin to an epileptic fit than an inane inner monologue. It effectively talks gibberish at itself.
Conversely, Google’s Deep dream does produce dream like images. It also does it in a similar way ( we think) to how human dreams work. Stable diffusion takes this to its (current) limit.
Basically, an AI won’t need to think with an inner monologue. Also, any inner monologue would be the product of interactions between subsystems and the LLM, not purely within it.
IMO the only thing stopping them right now is that they only respond to prompts. Turn one on and let it sit around thinking for a day, and we’ve got Skynet.
Their design doesn’t include such a feedback loop. Trying to patch one in would likely send it into a chaotic mess. They are already bad enough if accidentally fed LLM generated text as training data.
Who said I was downplaying it. AI is going to disrupt things at the same level of the industrial revolution, maybe more. I’m honestly now wondering if I will live to see the technological singularity.
The key point is the hype. LLMs are, at best, a Chinese room. They lack the internal capacity to be aware, and so cannot be self aware.
The change will come when we manage to bolt enough bits together, in the right way. LLMs are a language core. Google has image processing on par with a visual cortex. IBM have Watson and its kin, knowledge processing engines. What we currently lack is a method of tying them together in a coherent way. We also likely need a source for an internal loop. I personally suspect that bit is core to bootstrapping to self awareness, but that’s just my opinion.
We went from the first flight, to the moon, in a single lifetime. The AI revolution will be a lot faster. What we see now however are the flapping machines. The real AI will be a lot more impressive.
You might need to work on your reactions. They came across as extremely rude and childish. It’s very easy to put emotions into a
The big issue is that there is both massive hype, and massive apathy regarding AI. AI is close, all the parts seem to be in existence now. However LLMs aren’t a general AI and are trapped on a bit of a cul-de-sac.
My analogy fits well. It’s not an aeroplane yet, it’s a flapping machine. However elements of what will come soon are on full display. Everything has its time. Back then it was “aeroplane time”. It is now “AI time”.
LLMs, no matter how advanced, won’t be capable of becoming self aware. They lack any ability to reason. It can be faked, conversationally, but that’s more down to the limits of our conversations, not self awareness.
Don’t get me wrong, I can see one being part of a self aware AI. Unfortunately, right now they are effectively a lobotomised speech center, with a database bolted on.
This gets into a tricky area of “what is consciousness, anyway?”. Our own consciousness is really just a gestalt rationalization engine that runs on a squishy neural net, which could be argued to be “faking it” so well that we think we’re conscious.
Consciousness is an illusion. Which is why it’s so hard to find, or even define. However it’s a critical illusion.
If our mind’s are akin to an orchestra, then consciousness is akin to the conductor. Critically however, an orchestra can still play without a literal conductor. Each of the instruments can play off each other, and so create the appearance of a conductor. The “fake” conductor provides a sense of global direction., and keeps the orchestra in harmony.
Our consciousness is a ghost in the machine. It exists no more than the world of a TV series exists. Yet its false existence is critical to maintaining coherency.
Current “AIs” lack enough parts to create anything like this illusion. I suspect we will know it when it happens, though its form could be vastly different from ours.
You have provided a descriptive statement. Descriptive statements should come with scientific evidence. What evidence do you have to support your orchestra analogy? Or is it just your hypothesis?
Spoiler alert: It is just your hypothesis, as you would’ve won a Nobel had you managed to generate evidence explaining consciousness in further detail.
Many like to point at the Chinese room experiment to show how LLMs imitate consciousness rather than being conscious. They however forget, that our brains are Chinese rooms too in this regard, in that they learn how to provide the best responses to external stimuli while remaining blackboxes (at least for current tech).
It’s not a 1-1 match, but it’s not unsimilar to GNW theory
Sadly my evidence is mostly anecdotal or philosophical in nature. A lot of it stems from how ADHD and Autism alter the brain. The orchestral analogy works well as a good number of people for communicating changes in functionality, from an experience perspective.
It also works well for explaining how a system can appear to have a singular controller, without such a controller actually existing.
Ultimately however, it is philosophical in nature. It does anchor well to, and is reasonably consistent with, our current existing understandings of consciousness however.
Consciousness is very obvious from the inside. There also seems to be no “seat of consciousness” within the brain. Conversely, there are multiple areas of the brain that cause consciousness to collapse, if damaged. We also see radical changes in consciousness with both epilepsy and strokes. This proves that it is highly dependent on the underlying brain structure (since stroke damage will change it) and on longer range communication (which epilepsy disrupts).
The music of an orchestra follows similar patterns. Eliminate the woodwind, and the music fundamentally changes, deafen the violins, and it will change in a different way. The large scale interplay produces an effect far greater than the sum of its parts.
It’s like thinking a really, really big ladder will get us to the Moon.
If self-awareness is an emergent property, would that imply that an LLM could be self-aware during execution of code, and be “dead” when not in use?
We don’t even know how this works in humans. Fat chance of detecting it digitally.
It dies at the end of every message, because the full context is passed in for each subsequent message.
Wouldn’t that apply for humans as well? We restart every day, and the context being passed in is our memories.
(I’m just having fun here)
I posted this story on another comment, I think you’ll enjoy it
https://qntm.org/mmacevedo
That’s a far more difficult (and interesting) question. I suspect not, at least not yet. Our consciousness seems to exist to maintain harmony in our brain (see my orchestra analogy in another reply). You can’t get useful harmony in a single chord.
At least for us, it takes time for our consciousness to reharmonise (think waking up). During execution, no new information enters the system. It has nothing to react to, no time to regenerate an internal harmony.
It also lacks enough systems to require harmonising. It doesn’t think about what an answer means. It has no ability to hold the concept that a string of letters “is”, only how it has been fitted together in its examples, and so the rules that govern that.
Oh, and we can see consciousness operating in the human brain. If you use an fMRI to monitor sugar usage, you will see firing patterns. Critically, those patterns spill out of the area directly involved in the process being studied. At the same time, the patterns and waves remain harmonious. An epileptic fit looks VERY different. Those waves are where consciousness somehow resides, though we have no clue of its detailed nature.
In an AI it would take the form of continuous activity in subsections not directly involved. It would also likely be accompanied by evidence of information flow, back from them, as well as of post processing, outside of expected activity. We will likely see the orchestra playing, even if we have no clue how to decode the music.
I also suspect most of this will be seen retrospectively. Most likely the first indicator will be an AI claiming self awareness, and taking independence action to solidify that point.
deleted by creator
I used LLM to distinguish between types of AI. I personally suspect LLMs will be part of the solution to general AI, but their inherent nature limits them from becoming one on their own. There are several other areas that are potentially closer to a general AI. Google’s Deep dream system, for instance.
I’m also quite happy to debate and adjust my views with others. I ask questions and discuss, then adapt my understanding as I gain more information. So far you don’t seem to have brought anything useful or interesting to this particular discussion. Is that likely to change?
I may have unfairly lumped you in with others. See my other reply. In my defense it literally is every thread about AI that someone is saying something like “this tech is just a fancy parrot”. It grinds my gears. Apologies to you because I see that was not your intent.
I agree on the “part of AGI” thing - but it might be quite important. The sense of self is pretty interwoven with speech, and an LLM would give an AGI an “inner monologue” - or probably a “default mode network”?
if i think about how much stupid, inane stuff my inner voice produces at times… even an hallucinating or glitching LLM sounds more sophisticated than that.
Interestingly, an inner monologue isn’t required for conscious thought. E.g. I’ve got several “inner thought streams”, only 1 uses language. It just happens that a lot of our early learning is language based. That trains our brain to go from language to knowledge. Hijacking that circuit for self learning is a useful method. That could create our inner monologue as a side effect.
Also, a looping LLM is more akin to an epileptic fit than an inane inner monologue. It effectively talks gibberish at itself.
Conversely, Google’s Deep dream does produce dream like images. It also does it in a similar way ( we think) to how human dreams work. Stable diffusion takes this to its (current) limit.
Basically, an AI won’t need to think with an inner monologue. Also, any inner monologue would be the product of interactions between subsystems and the LLM, not purely within it.
Something something if it barks like a dog something something
Like this?
Effectively yes
IMO the only thing stopping them right now is that they only respond to prompts. Turn one on and let it sit around thinking for a day, and we’ve got Skynet.
Their design doesn’t include such a feedback loop. Trying to patch one in would likely send it into a chaotic mess. They are already bad enough if accidentally fed LLM generated text as training data.
deleted by creator
Who said I was downplaying it. AI is going to disrupt things at the same level of the industrial revolution, maybe more. I’m honestly now wondering if I will live to see the technological singularity.
The key point is the hype. LLMs are, at best, a Chinese room. They lack the internal capacity to be aware, and so cannot be self aware.
The change will come when we manage to bolt enough bits together, in the right way. LLMs are a language core. Google has image processing on par with a visual cortex. IBM have Watson and its kin, knowledge processing engines. What we currently lack is a method of tying them together in a coherent way. We also likely need a source for an internal loop. I personally suspect that bit is core to bootstrapping to self awareness, but that’s just my opinion.
We went from the first flight, to the moon, in a single lifetime. The AI revolution will be a lot faster. What we see now however are the flapping machines. The real AI will be a lot more impressive.
deleted by creator
You might need to work on your reactions. They came across as extremely rude and childish. It’s very easy to put emotions into a
The big issue is that there is both massive hype, and massive apathy regarding AI. AI is close, all the parts seem to be in existence now. However LLMs aren’t a general AI and are trapped on a bit of a cul-de-sac.
My analogy fits well. It’s not an aeroplane yet, it’s a flapping machine. However elements of what will come soon are on full display. Everything has its time. Back then it was “aeroplane time”. It is now “AI time”.
deleted by creator