How that preclude these models from being creative? Randomness within rules can be pretty creative. All life on earth is the result of selection on random mutations. Its output is way more structured and coherent than random noise. That’s not a good comparison at all.
Either way, generative tools are a great way for the people using to create with, no model has to be creative on its own.
A person sees a piece of art and is inspired. They understand what they see, be it a rose bush to paint or a story beat to work on. This inspiration leads to actual decisions being made with a conscious aim to create art.
An AI, on the other hand, sees a rose bush and adds it to its rose bush catalog, reads a story beat and adds to to its story database. These databases are then shuffled and things are picked out, with no mind involved whatsoever.
A person knows why a rose bush is beautiful, and internalises that thought to create art. They know why a story beat is moving, and can draw out emotional connections. An AI can’t do either of these.
A person is also very much adding rose bushes and story beats to their internal databases. You learn to paint by copying other painters, adding their techniques to a database. You learn to write by reading other authors, adding their techniques to a database. Original styles/compositions are ultimately just a rehashing of countless tiny components from other works.
An AI understands what they see, otherwise they wouldn’t be able to generate a “rose bush” when you ask for one. It’s an understanding based on a vector space of token sequence weights, but unless you can describe the actual mechanism of human thought beyond vague concepts like “inspiration”, I don’t see any reason to assume that our understanding is not just a much more sophisticated version of the same mechanism.
The difference is that we’re a black box, AI less so. We have a better understanding of how AI generates content than how the meat of our brain generates content. Our ignorance, and use of vague romantic words like “inspiration” and “understanding”, is absolutely not proof that we’re fundamentally different in mechanism.
You’re presupposing that brains and computers are basically the same thing. They are fundamentally different.
An AI doesn’t understand. It has an internal model which produces outputs, based on the training data it received and a prompt. That’s a different cathegory than “understanding”.
Otherwise, spotify or Youtube recommendation algorithms would also count as understanding the contents of the music/videos they supply.
An AI doesn’t understand. It has an internal model which produces outputs, based on the training data it received and a prompt. That’s a different cathegory than “understanding”.
Is it? That’s precisely how I’d describe human understanding. How is our internal model, trained on our experiences, which generates responses to input, fundamentally different from an LLM transformer model? At best we’re multi-modal, with overlapping models which we move information between to consider multiple perspectives.
A person painting a rose bush draws upon far more than just a collection of rose bushes in their memory. There’s nothing vague about it, I just didn’t feel like getting into much detail, as I thought that statement might jog your memory of a common understanding we all have about art. I suppose that was too much to ask.
For starters, refer to my statement “a person understands why a rose bush is beatiful”. I admit that maybe this is vague, but let’s unpack.
Beaty is, of course, in the eye of the beholder. It is a subjective thing, requiring opinion, and AIs cannot hold opinions. I find rose bushes beautiful due to the inherent contrast between the delicate nature of the rose buds, and the almost monstrous nature of the fronds.
So, if I were to draw a rose bush, I would emphasise these aspects, out of my own free will. I might even draw it in a way that resembles a monster. I might even try to tell a story with the drawing, one about a rose bush growing tired of being pkucked, and taking revenge on the humans who dare to steal its buds.
“Beauty”, “opinion”, “free will”, “try”. These are vague, internal concepts. How do you distinguish between a person who really understands beauty, and someone who has enough experience with things they’ve been told are beautiful to approximate? How do you distinguish between someone with no concept of beauty, and someone who sees beauty in drastically different things than you? How do you distinguish between the deviations from photorealism due to imprecise technique, and deviations due to intentional stylistic impressionism?
What does a human child draw? Just a rosebush, poorly at that. Does that mean humans have no artistic potential? AI is still in relative infancy, the artistic stage of imitation and technique refinement. We are only just beginning to see the first glimmers of multi-modal AI, recursive models that can talk to themselves and pass information between different internal perspectives. Some would argue that internal dialogue is precisely the mechanism that makes human thought so sophisticated. What makes you think that AI won’t quickly develop similar sophistication as the models are further developed?
Philosophical masturbation, based on a poor understanding of what is an already solved issue.
We know for a fact that a machine learning model does not even know what a rosebush is. It only knows the colours of pixels that usually go into a photo of one. And even then, it doesn’t even know the colours - only the bit values that correspond to them.
That is it.
Opinions and beauty are not vague, and nor are free will and trying, especially in this context. You only wish them to be for your argument.
An opinion is a value judgment. AIs don’t have values, and we have to deliberately restrict them to stop actual chaos happening.
Beauty is, for our purposes, something that the individual finds worthy of viewing and creating. Only people can find things beautiful. Machine learning algrorithms are only databases with complex retrieval systems.
Free will is also quite obvious in context: being able to do something of your own volition. AIs need exact instructions to get anything done. They can’t make decisions beyond what you tell them to do.
Trying? I didn’t even define this as human specific
I couldn’t have put it better myself. You’ve said lots of philosophical words without actually addressing any of my questions:
How do you distinguish between a person who really understands beauty, and someone who has enough experience with things they’ve been told are beautiful to approximate?
How do you distinguish between someone with no concept of beauty, and someone who sees beauty in drastically different things than you?
How do you distinguish between the deviations from photorealism due to imprecise technique, and deviations due to intentional stylistic impressionism?
Yeah, I know it doesn’t actually “see” anything, and is just making best guesses based on pre-gathered data. I was just simplifying for the comparison.
Interesting take on LLMs, how are you so sure about that?
I mean I get it, current image gen models seem clearly uncreative, but at least the unrestricted versions of Bing Chat/ChatGPT leave some room for the possibility of creativity/general intelligence in future sufficiently large LLMs, at least to me.
So the question (again: to me) is not only “will LLM scale to (human level) general intelligence”, but also “will we find something better than RLHF/LLMs/etc. before?”.
I’m not sure on either, but asses roughly a 2/3 probability to the first and given the first event and AGI in reach in the next 8 years a comparatively small chance for the second event.
Humans learn from other creative works, just like AI. AI can generate original content too if asked.
AI creates output from a stochastic model of its’ training data. That’s not a creative process.
What does that mean, and isn’t that still something people can employ for their creative process?
LLMs analyse their inputs and create a stochastic model (i.e.: a guess of how randomness is distributed in a domain) of which word comes next.
Yes, it can help in a creative process, but so can literal noise. It can’t “be creative” in itself.
How that preclude these models from being creative? Randomness within rules can be pretty creative. All life on earth is the result of selection on random mutations. Its output is way more structured and coherent than random noise. That’s not a good comparison at all.
Either way, generative tools are a great way for the people using to create with, no model has to be creative on its own.
They lack intentionality, simple as that.
Yup, my original point still stands.
How is intentionality integral to creativity?
Are you serious?
Intentionality is integral to communication. Creative art is a subset of communication.
I was asking about creativity, not art. It’s possible for something to be creative and not be art.
A person sees a piece of art and is inspired. They understand what they see, be it a rose bush to paint or a story beat to work on. This inspiration leads to actual decisions being made with a conscious aim to create art.
An AI, on the other hand, sees a rose bush and adds it to its rose bush catalog, reads a story beat and adds to to its story database. These databases are then shuffled and things are picked out, with no mind involved whatsoever.
A person knows why a rose bush is beautiful, and internalises that thought to create art. They know why a story beat is moving, and can draw out emotional connections. An AI can’t do either of these.
A person is also very much adding rose bushes and story beats to their internal databases. You learn to paint by copying other painters, adding their techniques to a database. You learn to write by reading other authors, adding their techniques to a database. Original styles/compositions are ultimately just a rehashing of countless tiny components from other works.
An AI understands what they see, otherwise they wouldn’t be able to generate a “rose bush” when you ask for one. It’s an understanding based on a vector space of token sequence weights, but unless you can describe the actual mechanism of human thought beyond vague concepts like “inspiration”, I don’t see any reason to assume that our understanding is not just a much more sophisticated version of the same mechanism.
The difference is that we’re a black box, AI less so. We have a better understanding of how AI generates content than how the meat of our brain generates content. Our ignorance, and use of vague romantic words like “inspiration” and “understanding”, is absolutely not proof that we’re fundamentally different in mechanism.
You’re presupposing that brains and computers are basically the same thing. They are fundamentally different.
An AI doesn’t understand. It has an internal model which produces outputs, based on the training data it received and a prompt. That’s a different cathegory than “understanding”.
Otherwise, spotify or Youtube recommendation algorithms would also count as understanding the contents of the music/videos they supply.
Is it? That’s precisely how I’d describe human understanding. How is our internal model, trained on our experiences, which generates responses to input, fundamentally different from an LLM transformer model? At best we’re multi-modal, with overlapping models which we move information between to consider multiple perspectives.
Humans have intentionality.
Emily M Bender can explain it better than I can
A person painting a rose bush draws upon far more than just a collection of rose bushes in their memory. There’s nothing vague about it, I just didn’t feel like getting into much detail, as I thought that statement might jog your memory of a common understanding we all have about art. I suppose that was too much to ask.
For starters, refer to my statement “a person understands why a rose bush is beatiful”. I admit that maybe this is vague, but let’s unpack.
Beaty is, of course, in the eye of the beholder. It is a subjective thing, requiring opinion, and AIs cannot hold opinions. I find rose bushes beautiful due to the inherent contrast between the delicate nature of the rose buds, and the almost monstrous nature of the fronds.
So, if I were to draw a rose bush, I would emphasise these aspects, out of my own free will. I might even draw it in a way that resembles a monster. I might even try to tell a story with the drawing, one about a rose bush growing tired of being pkucked, and taking revenge on the humans who dare to steal its buds.
All this, from the prompt “draw a rose bush”.
What would an AI draw?
Just a rose bush.
“Beauty”, “opinion”, “free will”, “try”. These are vague, internal concepts. How do you distinguish between a person who really understands beauty, and someone who has enough experience with things they’ve been told are beautiful to approximate? How do you distinguish between someone with no concept of beauty, and someone who sees beauty in drastically different things than you? How do you distinguish between the deviations from photorealism due to imprecise technique, and deviations due to intentional stylistic impressionism?
What does a human child draw? Just a rosebush, poorly at that. Does that mean humans have no artistic potential? AI is still in relative infancy, the artistic stage of imitation and technique refinement. We are only just beginning to see the first glimmers of multi-modal AI, recursive models that can talk to themselves and pass information between different internal perspectives. Some would argue that internal dialogue is precisely the mechanism that makes human thought so sophisticated. What makes you think that AI won’t quickly develop similar sophistication as the models are further developed?
Philosophical masturbation, based on a poor understanding of what is an already solved issue.
We know for a fact that a machine learning model does not even know what a rosebush is. It only knows the colours of pixels that usually go into a photo of one. And even then, it doesn’t even know the colours - only the bit values that correspond to them.
That is it.
Opinions and beauty are not vague, and nor are free will and trying, especially in this context. You only wish them to be for your argument.
An opinion is a value judgment. AIs don’t have values, and we have to deliberately restrict them to stop actual chaos happening.
Beauty is, for our purposes, something that the individual finds worthy of viewing and creating. Only people can find things beautiful. Machine learning algrorithms are only databases with complex retrieval systems.
Free will is also quite obvious in context: being able to do something of your own volition. AIs need exact instructions to get anything done. They can’t make decisions beyond what you tell them to do.
Trying? I didn’t even define this as human specific
I couldn’t have put it better myself. You’ve said lots of philosophical words without actually addressing any of my questions:
The way you describe how these models work is wrong. This video does a good job of explaining how they work.
Yeah, I know it doesn’t actually “see” anything, and is just making best guesses based on pre-gathered data. I was just simplifying for the comparison.
LLM AI doesn’t learn. It doesn’t conceptualise. It mimics, iterates and loops. AI cannot generate original content with LLM approaches.
Interesting take on LLMs, how are you so sure about that?
I mean I get it, current image gen models seem clearly uncreative, but at least the unrestricted versions of Bing Chat/ChatGPT leave some room for the possibility of creativity/general intelligence in future sufficiently large LLMs, at least to me.
So the question (again: to me) is not only “will LLM scale to (human level) general intelligence”, but also “will we find something better than RLHF/LLMs/etc. before?”.
I’m not sure on either, but asses roughly a 2/3 probability to the first and given the first event and AGI in reach in the next 8 years a comparatively small chance for the second event.