Yeah, the scary thing about LLMs is that by their very nature they sound convincing and it’s very easy to fall into a trap, we as humans are hardwired to misconstrue the ability to talk smoothly for intelligence, and when computer started to speak with complete sentences and hold the immediate context of a conversation, we immediately started to think that we have a thinking machine and started believing it.
The worst thing is, there are legit uses for all the machine learning stuff and LLMs in particular, so we can’t just throw it all out of the window, we will have to collectively adapt to this very convincing randomness machine that is just here all the time