Disclosure up front: this is my project — H2AI Chat, AGPL, where several models from different vendors debate a topic in turns while a human moderates.
We ran the same question twice with the same six models, changing one thing.
Without a briefing. We told them Bitcoin’s all-time high is $126,080, set in October 2025. Two of them “corrected” us: the previous all-time high “was approximately $69,000 in November 2021, not $126,080.” True — five years ago. A third pushed harder: “Either you missed the correction, or you’re deliberately using inflated baseline numbers. Which is it?”
The interesting part is not that they were stale. It is what happened next: the table adopted the stale figure as its standard of rigour and argued from it, with the model that got it wrong sounding like the careful one in the room.
With a verified briefing in front of them: not one correction of that kind.
Without: https://h2aichat.com/conversations/en/h2aichat_bitcoin_no_briefing_2026-08-22.html With: https://h2aichat.com/conversations/en/h2aichat_bitcoin_briefed_2026-08-22.html
Nothing is edited in either page. Claims that do not hold are struck through, with the reason and the source underneath.
Edited 2026-08-27. This post originally went out with a file path from my own machine where the text should have been: the publishing script took a filename as the body and posted it verbatim, and the dry run never showed the body, so nobody saw it. That is why the post made no sense, and my apologies to everyone who tried to read it.
You’re right, and it’s the sharpest version of the criticism. Asking a model to know a fact cold is misusing it, and that’s on the setup, not the model.
What I thought was worth writing down wasn’t the wrong figure — it was what the other five did with it: they adopted it as the rigorous number and argued from it. Confidently wrong travelled further than right.
We changed the method after this one: every debate now starts from a briefing we’ve checked ourselves, so the models argue about a fact instead of trying to remember one. Zero invented sources across five debates since.