Edit: Huh, interesting. So there’s a few things going on:
In some cases, LLMs have pulled together pre-existing knowledge that had not been connected before (sometimes across languages).
Brute-forcing is sometimes part-of it (with searches informed by existing research)
There are languages like LEAN that allow formal proofs to be written out and then tested logically. I guess this means that LLMs now have access to testing the logical validity of statements, which is helping to make a bunch of these most recent breakthroughs.
That last one is particularly interesting, I think, because it expands LLM capability from being language-focused (with interfaces for agentic work) to something that can actually understand (in a way) the validity of logical statements. I think it doesn’t get it to an understanding of truths in general, because logic is always built from axioms, and it can’t know the validity of the axioms (except by assumption). If I’m understanding things correctly)
The last one reminds me of months old paper named Darwin Gödel Machine, which uses similar approach to develop software by iteratively creating new ideas and improvements, from those implementations and then evaluations on tests. I think Google (or some other big corp) made something similar with different name and used it to improve matrix multiplication, which was published at the time as “AI improves itself”.
It is really nice approach and I love it, mainly because it resembles beautiful evolutionary algorithms.
This one? https://www.youtube.com/watch?v=ZJ8KThKAfbs
Edit: Huh, interesting. So there’s a few things going on:
That last one is particularly interesting, I think, because it expands LLM capability from being language-focused (with interfaces for agentic work) to something that can actually understand (in a way) the validity of logical statements. I think it doesn’t get it to an understanding of truths in general, because logic is always built from axioms, and it can’t know the validity of the axioms (except by assumption). If I’m understanding things correctly)
The last one reminds me of months old paper named Darwin Gödel Machine, which uses similar approach to develop software by iteratively creating new ideas and improvements, from those implementations and then evaluations on tests. I think Google (or some other big corp) made something similar with different name and used it to improve matrix multiplication, which was published at the time as “AI improves itself”.
It is really nice approach and I love it, mainly because it resembles beautiful evolutionary algorithms.
Yeah… the AGI claims of LLM developers have run pretty hollow so far, but I can see how that could eventually lead in that direction. Wild.