This is interesting as it runs counter to what many people think about current AI. Its performance seems directly linked to the quality of the training data it has. Here the opposite is happening; it has poor training data and still outperforms humans. It’s not surprising the humans would do badly in this situation too; it’s hard to keep up to date on things that you may only encounter once or twice in your entire career. It’s interesting to extrapolate from this observation as it applies to many other fields.
I mean, recognition is literally the task that is always used for intro to machine learning. From facial recognition and other biometric, handwriting, object recognition. It isn’t a surprise that “AI” is able to outperform humans in this task since sometimes AI can pick up features that are too subtle for us to notice. The problem is LLM being hailed as the truth machine or AGI. LLM to NLP is what CNN and GAN is to image processing tasks.
This is interesting as it runs counter to what many people think about current AI. Its performance seems directly linked to the quality of the training data it has. Here the opposite is happening; it has poor training data and still outperforms humans. It’s not surprising the humans would do badly in this situation too; it’s hard to keep up to date on things that you may only encounter once or twice in your entire career. It’s interesting to extrapolate from this observation as it applies to many other fields.
One of the authors of the paper goes into more detail on Twitter.
I mean, recognition is literally the task that is always used for intro to machine learning. From facial recognition and other biometric, handwriting, object recognition. It isn’t a surprise that “AI” is able to outperform humans in this task since sometimes AI can pick up features that are too subtle for us to notice. The problem is LLM being hailed as the truth machine or AGI. LLM to NLP is what CNN and GAN is to image processing tasks.