Also not a researcher, but I also believe hallucinations are simply the artifact of being able generate responses that aren’t pure reproduction of training data. Aka, the generalization we want. The problem is we have something that generalize without the ability to judge what it thinks of.

It will in my opinion never go away, but I’m sure it can be improved significantly.