One of the use cases I thought was reasonable to expect from ChatGPT and Friends (LLMs) was summarising. It turns out I was wrong. What ChatGPT isn’t summarising at all, it only looks like it…
People always assume that generative AI (and technology in general) will continue improving at the same pace it always has been. They always assume that there are no limits in the number of parameters, that there’s always more useful data to train it on, and that things like physical limits in electricity infrastructure, compute resources, etc., don’t exist. In five years generative AI will have roughly the same capability it has today, barring massive breakthroughs that result in a wholesale pivot away from LLMs. (More likely, in five years it’ll be regarded similarly to cryptocurrency is today, because once the hype dies down and the VC money runs out the AI companies will have to jack prices to a level where it’s economically unviable to use in most commercial environments.)
People always assume that generative AI (and technology in general) will continue improving at the same pace it always has been. They always assume that there are no limits in the number of parameters, that there’s always more useful data to train it on, and that things like physical limits in electricity infrastructure, compute resources, etc., don’t exist. In five years generative AI will have roughly the same capability it has today, barring massive breakthroughs that result in a wholesale pivot away from LLMs. (More likely, in five years it’ll be regarded similarly to cryptocurrency is today, because once the hype dies down and the VC money runs out the AI companies will have to jack prices to a level where it’s economically unviable to use in most commercial environments.)
To add to this, we’re going to run into the problem of garbage in, garbage out.
LLMs are trained on text from the internet.
Currently, a massive amount of text on the internet is coming from LLMs.
This creates a cycle of models getting trained on data sets that increasingly contain large sets of data generated by older models.
The most likely outlook is that LLMs will get worse as the years go by, not better.