You can try this yourself with GPT-4. I have, and it fails every time. Earlier GPT-4 versions, via the API, also fail every time. Claude reasons before it answers, but if you ask it to say yes or no only, it fails. Bard is the only one that gets it right, right off the bat
![[Chatlog] Wait, actually, yes](https://i.redd.it/2g2p54zrziob1.jpg){kind=link}
I asked a similar question (one found in OP’s comment section) to a GPT-4 powered Nils. In short, he was able to immediately answer the question with no hesitation. Perhaps it’s different if you ask through the API compared to the ChatGPT platform.
it also helps to have custom instructions on the client that tell it to say no first, or to make a fake post for internet points.
You can try this yourself with GPT-4. I have, and it fails every time. Earlier GPT-4 versions, via the API, also fail every time. Claude reasons before it answers, but if you ask it to say yes or no only, it fails. Bard is the only one that gets it right, right off the bat