The AI boom has turned the standard profit margin model on its head, according to Apollo Chief Economist Torsten Slok—and it’s making the industry’s growth unsustainable.
That is where the evidence points. Now, I don’t want to oversell it: “where the evidence points” is wildly different than “exactly how it works”.
We have an emergent property that we don’t understand, but we can reliably increase the functionally and complexity of that emergent property as a function of training data and available compute. Does that mean that there isn’t some threshold where that stops working? No, there certainly could be a point where throwing more information and compute has no effect. We just don’t know. However, so far, there is no evidence such a barrier exists, and everyone is racing to find out.
I think that is where the hopes and hype point. The evidence, that which is gathered through controlled studies, points to an upper limit to this technology that does have emergent properties, but not ones that amount to cognition. That evidence also points to other hurdles, such as cognitive damage to the user and context windows nowhere near that of even a simpleminded creature, let alone a sapient one like a human.
The core problem is that these models are fixed; they are the same on day 1000 as they were on day 1. All their “learning”, as it were, happens in training before it is released. Everything it appears to learn after that date is contained in the rolling context window. Since they already have literally all the RAM they can get their hands on and are still at least an order of magnitude away from where they need to be on that, this technology either can’t do it or, at best, is so inefficient an approach that it can’t be done with all the planet’s resources.
Sometimes, especially in abstract constructions like software, you can start down the wrong path early and have to start over, because there is no path from where you are to where you need to get. In this case, they may have done that to the extreme, blinded by the lucrative prospects.
As far as I am aware, AGI does not imply sapience, consciousness, or whatever. You seem to believe it does. A high level Google search seems to suggest that I’m correct. Is there some reason you are lumping sapience with AGI?
I’m not sure if the rest of your comment presupposes this, so I’ll wait to comment on where I think you’re mistaken.
I dont believe it requires sapience; that is what the marketers are saying. The AI boom (and many historic boom cycles) is predicated on marketing and sentiment, not facts and data. That is why they always pop; the facts dont back up the hype.
The fact that your search results turn up results that align with the marketing is just the marketing working.
Look–neither of us are data scientists, but we do have eyes. If this is working, where’s the company with runaway success creating unimagined leaps in productivity and technology? We’re pouring a whole planet’s with of resources in and nothing much is coming out. If we spent this much on world hunger, everyone would be obese by now.
I mean, great. But why bring up obesity, which is often a consequence of poverty? Also, it’s not just about how much food people can obtain. There are so many other factors involved in body shape and size.
I dont believe it requires sapience; that is what the marketers are saying. The AI boom (and many historic boom cycles) is predicated on marketing and sentiment, not facts and data. That is why they always pop; the facts dont back up the hype.
I think there is some confusion. I do not believe AGI implies sapience, nor does Google, and now it seems that you don’t either. So who is discussing sapience?
If this is working, where’s the company with runaway success creating unimagined leaps in productivity and technology?
Hypothetically speaking, what do you think this would look like?
The marketers are not using the word “sapience”, as that word would require defining for the layman. They are claiming AI will usher in an era of human prosperity by taking over our work and big decisions to do them more efficiently–tasks that sound to almost everyone to require advanced decision-making and thought, the hallmarks of sapience and cognition. This is largely besides the point, though.
I think that if this were successful, you would see at least some early adopting companies coming up aces. Some company would have double the productivity with half the workforce, and use that to absolutely devour their market segment by undercutting everyone else on price because their operating costs are so much lower than the competition. I’d see a software company adopt breakneck software release cycles with substantial, material stability and performance improvements with each. I’d see some company anywhere suddenly begin to outperform their former selves and be able to convincingly point to AI as where their improvements came from. The only one like that I can think of is NVidia, who are rich strictly because they’re selling shovels in the gold rush.
What it would look like is layoffs from a company without a change in output/productivity. Notably, we’re not at AGI yet, so I wouldn’t expect sci-fi levels of change.
That is one thing it could look like which is captured in my examples. We dont see any company like that. Remember that stock valuation is not, in the present day, correlated to any business metric. There are companies with huge valuations that have never netted $1 (like these AI companies, for example). So, even in your narrower example, there are no companies I have heard of that meet its criteria.
There would necessarily be a lag time between layoffs and unchanged productivity. And even then, if we’re being fair, that’s not conclusive evidence of AI being useful; it could just be that they had more employees than they needed.
My point is more that I’m not sure we’d see an obvious sign that AI was changing the market yet.
That is where the evidence points. Now, I don’t want to oversell it: “where the evidence points” is wildly different than “exactly how it works”.
We have an emergent property that we don’t understand, but we can reliably increase the functionally and complexity of that emergent property as a function of training data and available compute. Does that mean that there isn’t some threshold where that stops working? No, there certainly could be a point where throwing more information and compute has no effect. We just don’t know. However, so far, there is no evidence such a barrier exists, and everyone is racing to find out.
I think that is where the hopes and hype point. The evidence, that which is gathered through controlled studies, points to an upper limit to this technology that does have emergent properties, but not ones that amount to cognition. That evidence also points to other hurdles, such as cognitive damage to the user and context windows nowhere near that of even a simpleminded creature, let alone a sapient one like a human.
The core problem is that these models are fixed; they are the same on day 1000 as they were on day 1. All their “learning”, as it were, happens in training before it is released. Everything it appears to learn after that date is contained in the rolling context window. Since they already have literally all the RAM they can get their hands on and are still at least an order of magnitude away from where they need to be on that, this technology either can’t do it or, at best, is so inefficient an approach that it can’t be done with all the planet’s resources.
Sometimes, especially in abstract constructions like software, you can start down the wrong path early and have to start over, because there is no path from where you are to where you need to get. In this case, they may have done that to the extreme, blinded by the lucrative prospects.
As far as I am aware, AGI does not imply sapience, consciousness, or whatever. You seem to believe it does. A high level Google search seems to suggest that I’m correct. Is there some reason you are lumping sapience with AGI?
I’m not sure if the rest of your comment presupposes this, so I’ll wait to comment on where I think you’re mistaken.
I dont believe it requires sapience; that is what the marketers are saying. The AI boom (and many historic boom cycles) is predicated on marketing and sentiment, not facts and data. That is why they always pop; the facts dont back up the hype.
The fact that your search results turn up results that align with the marketing is just the marketing working.
Look–neither of us are data scientists, but we do have eyes. If this is working, where’s the company with runaway success creating unimagined leaps in productivity and technology? We’re pouring a whole planet’s with of resources in and nothing much is coming out. If we spent this much on world hunger, everyone would be obese by now.
@badgermurphy @joe
WTF is up with that last line? :(
deleted by creator
@badgermurphy
I mean, great. But why bring up obesity, which is often a consequence of poverty? Also, it’s not just about how much food people can obtain. There are so many other factors involved in body shape and size.
You could just say, “Feed the entire world.” :/
I think there is some confusion. I do not believe AGI implies sapience, nor does Google, and now it seems that you don’t either. So who is discussing sapience?
Hypothetically speaking, what do you think this would look like?
The marketers are not using the word “sapience”, as that word would require defining for the layman. They are claiming AI will usher in an era of human prosperity by taking over our work and big decisions to do them more efficiently–tasks that sound to almost everyone to require advanced decision-making and thought, the hallmarks of sapience and cognition. This is largely besides the point, though.
I think that if this were successful, you would see at least some early adopting companies coming up aces. Some company would have double the productivity with half the workforce, and use that to absolutely devour their market segment by undercutting everyone else on price because their operating costs are so much lower than the competition. I’d see a software company adopt breakneck software release cycles with substantial, material stability and performance improvements with each. I’d see some company anywhere suddenly begin to outperform their former selves and be able to convincingly point to AI as where their improvements came from. The only one like that I can think of is NVidia, who are rich strictly because they’re selling shovels in the gold rush.
What it would look like is layoffs from a company without a change in output/productivity. Notably, we’re not at AGI yet, so I wouldn’t expect sci-fi levels of change.
That is one thing it could look like which is captured in my examples. We dont see any company like that. Remember that stock valuation is not, in the present day, correlated to any business metric. There are companies with huge valuations that have never netted $1 (like these AI companies, for example). So, even in your narrower example, there are no companies I have heard of that meet its criteria.
There would necessarily be a lag time between layoffs and unchanged productivity. And even then, if we’re being fair, that’s not conclusive evidence of AI being useful; it could just be that they had more employees than they needed.
My point is more that I’m not sure we’d see an obvious sign that AI was changing the market yet.