Computer pioneer Alan Turing’s remarks in 1950 on the question, “Can machines think?” were misquoted, misinterpreted and morphed into the so-called “Turing Test”. The modern version says if you can’t tell the difference between communicating with a machine and a human, the machine is intelligent. What Turing actually said was that by the year 2000 people would be using words like “thinking” and “intelligent” to describe computers, because interacting with them would be so similar to interacting with people. Computer scientists do not sit down and say alrighty, let’s put this new software to the Turing Test - by Grabthar’s Hammer, it passed! We’ve achieved Artificial Intelligence!
Hence the reason it’s not a real intelligence (yet) even a goldfish can do problem solving without first having to be equipped with god like levels of prior knowledge about the entire universe.
Current LLM’s aren’t that stupid. They do have limited learning. You give it a question, tell it where it’s wrong and it will remember and change all future replies with the new information you give it. You certainly can’t ask a goldfish to write a c program that blinks an led on a microcontroller. I have used it to get working programs to questions that were absolutely nowhere on the internet. So it didn’t just copy/paste something found.