This article is a thinly veiled advertisement for ChatGPT. It’s actually about some guy called Bloom who used ChatGPT to build a web site.
I don’t think that’s a fair take. The website Bloom built is useful background for why this is big news, and how we got here. But this isn’t just about Bloom:
And, on May 20, OpenAI shared a solution (opens a new tab) to the unit distance problem, along with a blog post (opens a new tab) explaining the work and a companion paper (opens a new tab) that featured nine world-class mathematicians commenting on the correctness of the proof and the importance of what had been done (as well as presenting a streamlined human version of the result). Mathematicians had generally believed that Erdős’ conjecture — about how many evenly spaced points can be placed on a plane — was correct. To general surprise, OpenAI’s internal model found a counterexample. To do so, it had found a sophisticated way to use tools from an area of math called algebraic number theory. As Jacob Tsimerman (opens a new tab) of the University of Toronto wrote in the companion article, “This is a really impressive piece of work. … It is definitely an intimidating construction.”
That’s an impressive, important result.
At this point, I don’t really understand the point of reductive arguments like “it’s just a next-token predictor” in regards to LLMs. Even claims of “it’s not useful” (not even limited to “not useful to me”, just flat out “not useful”). There’s always the response of comparing any argument to humans, e.g. “humans can generate bullshit too”, but even aside from that, the proof is in the pudding, so to speak. AI is doing interesting things.
There’s downsides like power usage and centralized control by capitalists, but IMO those are problems to solve, not reasons to pretend AI is just hype.
Ai is a catch all term for everything from tech parsing throuh maths problems to tech undressing family photos on twitter.
The tech isnt useless, it is wildly overstated. It is wildly disproportionate to what has been invested and the damages it causes.
I can’t really square “wildly overstated” with the progress made so far with these models. Is there a specific claim you’re thinking of that can be evaluated? Otherwise it feels like nutpicking, where you pick some audacious claim that nobody/barely anyone agrees with, and then go hard against that, ignoring the more reasonable positions.
Just looking at this article, how much do you think is worth investing in “AI” (however you want to define it) that can find novel mathematical results? Isn’t that good for humanity and something we should be spending a solid amount of resources on?
Personally, I don’t find much use in talking purely about the damages or harms of AI. It’s often used to just dismiss it entirely, without trying to consider at all if those can be solved or mitigated. In other words, saying “AI is bad because X” isn’t worth discussion. “How do we solve X caused by AI” is much more interesting.
EDIT: Funnily enough, I just got an email from the EFF that captures my position pretty well:
The important thing about a technology isn’t just what it does: it’s who it does it for and who it does it to. Cory Doctorow and EFF Executive Director Nicole Ozer are tackling what needs to happen now to ensure AI actually works for everyone, not just those in power.
Join us on Wednesday, August 12 at 10:00 am Pacific for the latest installment of our EFFecting Change Livestream Series: Who the Machine Serves. AI can help or harm people, and EFF was created to make sure it helps. We are at a critical juncture to ensure that AI is developed and used in ways that respect fundamental rights and works for those who build it, use it, and are affected by it. Find out more and bring your questions for this livestream with live Q&A.
EFFecting Change Livestream Series: Who the Machine Serves
Wednesday, August 12
10:00 am - 11:00 am Pacific - Check Local Time
Livestream followed by Q&AI like your focus here on what is more interesting. What I heard about the Erdős problem solution (caveat: I am no expert there) is not that a human couldn’t solve it, but that they didn’t bother.
My take-away from that is that the problem was either not interesting at all - at least, not interesting enough to attract any human’s interest to even explore the space a bit more than has already been done - or else that the expectation was that the amount of resources that would need to be expended would not be worthwhile. Either way takes me along interesting lines of thought.
The most negative one, also a dead end so let’s get it out of the way first, is that once LLM tokens start to cost appreciable amounts of money, then realistically the same thing is going to happen again and again with “AI”, just as it previously did for humans. So… one data point does not make a trend, and if anything even shows suggestions of being an exception proving the rule?
The more positive take there is that this is evidence of how LLMs can be most useful. Yes, a human could have done it, but they did not bother - just like a human could spell or add 99+13 together, but a computer can go through such so much more quickly. However… adding 99+13 or spelling words correctly does not make an LLM sentient, nor help it qualify for the misnomer “AI”, which conjures up the suggestion of an AGI. People are literally throwing around the words “genius” to describe this mathematical theorem proof - which seems patently false. Even if you alter that to a “senile genius”, it still does not seem to apply? Again, a human COULD have done this, they simply DID NOT BOTHER to… so how does that qualify the proof as “genius”?
Intriguing? Definitely! Interesting? Since the source is a computer, most definitely! Impressive? Still yes! Important? Well, in the sense of a milestone reached, sure why not! But… “genius”, even a senile one? No… nor AGI either, and therefore also lacking justification that the LLM has finally reached a level to truly qualify as being called an “AI”, except in the most base, crass manner - like is my calculator also an “AI”? It can add though!!! And modern AI is at least aware of 2 out of the 3 "r"s in the word strawberry… which again is not “genius” level nor anything remotely close.
It is these lies then that obscure the real truths. Like a car salesman trying to make a sale and making shit up, behind which you might even legitimately want the vehicle, but it detracts from the process of you finding the best one for you, when you have to expend the additional effort to not only do that, but to do it while simultaneously countering the disinformation being thrown at you too.
Ultimately it seems to come down to me to this one point: LLMs are not AGI yet, nor are they geniuses, and whether they ever will be or not remains an open question. This theorem proof seems a minor milestone? And “proves” (hehe) that theorem proving at a low level can done by future LLMs. However, I was doing similar tasks back in the oughts with theorem-solving computer programs, and while we’ve come a long way, this seems by no means a major leap forwards in human historical terms. Maybe I’m wrong and we’ll look back and see it this way, but even if we took that as a given, it still does not follow that this is a proven fact from our current perspective.
And every time I follow through on yet another exaggerated claim like this - I think we both have said that while the claim is REAL, it has also been exaggerated by its proponents? (though haven’t quite pinned down by how much yet) - it turns out to be a LIE. One day… it won’t be. But until then, I understand people’s frustrations when 99.9999% of the statements ultimately result in them proven to have been falsehoods, hence they distrust any further statements, definitely from the same sources but perhaps it is a human bias yet also it makes sense to distrust those on the same topic as well.
One day a real AGI is going to be created. That will be exciting!! And before that AGI can run, it will walk, and before that it will crawl, and before that… well anyway, I have yet to see evidence that we have suddenly skipped over all of the intermediate steps and arrived already at the level of “
stablesenile genius”.EXTRAORDINARY CLAIMS REQUIRE EXTRAORDINARY EVIDENCE
Which simply has not been forthcoming. And with all the billions having been invested so far - and I saw a title of an article even suggesting that people may offer a trillion dollar evaluation? (I don’t believe it, but still it seems relevant that this is the level of money that we are talking about being thrown around here) - even with ALL of that, I would think that if they TRULY had a “genius”, then they would… what?
Either use that genius AI, without sharing with anyone, claiming bankruptcy to get out of the debts and that the entire AI field was incorrect, thus poisoning the well and thereby buying them time to extend their head start (this is literally the origin story behind Facebook btw, at least according to The Social Network film, where Mark Zuckerberg lied to his first investors to halt their competition - that wasn’t the actual Facebook btw, it was its precursor while he was still enrolled in Harvard).
Or show that proof to the world? Thereby justifying all of the investment so far, hence ensuring it’s continuation into the future, to the tune of at the very least hundreds of billions if not literally trillions of dollars? Unless the proof is too complex for people to see the benefits? At which point we go back not quite precisely to the above scenario, but what then would be the functional difference between them?
And even if all of this were false, and LLMs truly were as good as the techbros say, what even would the technology be used for? Enriching stockbrokers further? Perhaps bankers too? And owners of rental properties? It’s what literally all other technological advances in the last 5 or so years have done, so why would these be any different?
Open access models might truly offer something different though! Yet… wasn’t this theorem proof performed by a closed-source one? I suppose one line of thought is that if a closed one can do it then an open one could too.
But it’s so difficult to see what is true or not, when primarily the story is being told from the perspective of the liars. So thank you for the contrasting viewpoints here, since Cory Doctorow seems to not be one of them, and we need people fighting the good fight out there. Which presumes that there even is a fight that exists, and that the whole entire thing is not merely a made-up fiction by the techbros. To clarify, I do not mean to say that the idea that an AGI will eventually emerge is a fiction, but rather that one has already done so, hence give me money please ⁉️
not that a human couldn’t solve it, but that they didn’t bother
I’d agree, but with the caveat that that’s still impressive and useful. There’s kind of an LLM of the gaps thing going on, where LLMs obviously can’t produce novel mathematical results, right up until they can. There’s limitations and caveats, and one shouldn’t trust most anything tech ceos say, but the models continue to progress in capabilities. This is the stuff of science fiction a few years ago.
One day a real AGI is going to be created.
IMO this is real AGI. It’s just not ASI. It’s a general intelligence because it’s capable of handling a huge variety of tasks fairly well without needing to be trained specifically for each one. It’s clearly not as smart as humans, and thus isn’t an ASI. The specific terms I’m not really attached to, moreso the point is that we need more specific ways of talking about intelligence.
right up until they can
Absolutely, this right here. Except they haven’t yet. So one day they will be at genius level. But this is not that day. Maybe literally tomorrow? More likely LLMs will never be sufficient on their own, at least in the more neural network sense, and probably in the near future a more mathematical & counting functionality will be grafted onto the LLM, so that a question like “what is 1+1” will at least have awareness of the fact that computationally speaking the answer would be 2, even if the actual answer given is something else, like “little Bobby, you told me that you would consent to going to bed in 10 minutes… but that was 25 minutes ago, and you asked me to do just one more… and we did five more” - you know, true AGI stuff.
Which when it comes will be astonishing. But for now it makes astonishing mistakes, yet techbros want all of the water and electricity and to keep the profits while off-loading all the costs to society at large now, even though the remotest possibility of a true AGI has yet to be demonstrated can come from an LLM - especially alone without other counterbalancing components.
I don’t trust anyone who calls these theorem-proving models as “geniuses”, senile or otherwise. An LLM yes, an “AI” very much controversially (where the outcome depends upon power and politics, not any kind of search for realism to represent Truth), an intriguing contribution definitely yes, an important one the jury is still our on how useful the result will ultimately be but I preemptively concede that it at least could be, but an AGI… can it do anything other than predict, based on its training data?
It can shuffle, mixing and matching existing things according to a weighted fashion and in a manner such that we humans may find the end product useful (at least sometimes, although a LOT ends up being discarded as well and that is something that I find missing from most discussions - e.g. if a googleplex of monkeys hammering away on a keyboard can produce a Shakespearian play, buried amongst googles of nonsense words, does that make the monkeys “smart”?), but can an LLM think? That is what the “I” stands for, after all?
At which point I’ve wrapped around to agreeing with you, not that we currently have an AGI bc I think we do not yet (definitely not a “genius” one, senile or otherwise), but that in order to even answer that we would have to know what that “I” even stands for in the first place. Critical thinking? Ability to count and do computations? Articulation of internal processes? And so on.
The trick is to define “think”. Nobody has a good definition that isn’t circular, because we don’t really know enough about how the brain works to come up with an objective, testable definition. It all loops around endlessly, “think” to maybe “conceive” or “ponder” or “know” or “aware” or “conscious” or “sentient”, so on and so on. I think LLMs think in the same way that planes and bees both “fly”, even if it’s very different mechanisms 🤷
I agree that the models aren’t geniuses, and we should critically evaluate them, especially whenever OpenAI or other such companies make claims, and all that. I just don’t really care for arguing about “is it truly thinking”, but maybe that’s just the engineer side of me:
https://www.smbc-comics.com/?id=1879
Going off slightly on a tiny tangent: if a human is not merely a set atoms (which get continually replaced anyway) but the PATTERN of them, then a molecular-level copy is essentially the same person. Or at least as functionality indistinguishable from them as makes no difference. Yes obviously the one who existed first “came first”, which is a tautology hence uninteresting.
I think the issue with LLMs, for most people who aren’t hip-deep into the techbro culture that seems to be basically worshiping the idea of playing the role of a god to create a new technological lifeform, is not whether they think or not, but whether they are USEFUL to them or not. And there’s the rub: when an “AI” can search the Internet for you - especially nowadays when it is so enshittified, burying information behind multiple clicks, requests to sign-up to newsletters, JavaScript and even CSS interactivity that fights you at every step of the way trying to glean information from a website - and return something useful, like a recipe to cook food, then people enjoy using it! Until it turns out that the recipe is fatal to humans that consume the product - e.g. when it contains glue in it (arguably producing the best picturesque foodstuff products, suitable for visual presentation rather than edible consumption), or razor blades (that one representing straight-up poisoning the well scenarios, but since those can often represent sarcasm - even in material that long predates the emergence of LLMs hence was genuinely meant for a fellow human to read - it is not always so easy to detect).
My primary issue lies with how modern LLMs offers the form of an answer, while cheapening out on the substance. It thus flips the standard mode of evaluation, making it more rather than less difficult for someone to use it to find information. e.g. if you come across a page on the internet that is full of spelling errors, then the chances of its content being accurate in spite of that is next to nil. However, ChatGPT has straight up told people to do things that would literally kill them. I am used to an older model of computation where if you ask a computer what 1+1 is it will deterministically always tell you “2” (aside from issues such as hardware failure, including running low on battery power supply), whereas LLMs simply work according to an entirely different model, where the answer might sometimes be 3, or 1 again, or the message to KYS, etc. And I don’t think that most people - especially literal children - realize the switch. They are bullshit generators - which I mean to say not only that in the derogatory sense, but more foundationally that is literally what they were designed to be, in being able to provide an answer in a certain format and when an answer is not forthcoming then to simply make shit up… by design.
So I would guess that the “is it thinking” questions might come more from those techbros who are in love with the idea of technological emergence of sentience, whereas most common folk simply want to be able to find information on the internet, especially now that SEOs have made Google notably less useful at that task. And for them, hearing about whether a tool is at the “genius” level or not is a proxy for how much they can place trust in the outcome of a query. Which is to say, not much - a position that Sam Altman is expressing himself so by no means a niche or even exclusively outsider concern.
