I was recently reading Ben Aaronvitch’s ‘Rivers of London’ and I was struggling to follow the plot. So I engaged in discussions with both ChatGPT and Gemini and both really struggled to not make lots of mistakes. Both AIs would regularly give inaccurate info about major plot points, and when I questioned them about those issues, they would apologize profusely and then try to continue on. I basically had to coach them through to get them to remember important points of the plot! They also struggled to not reveal spoilers past a certain chapter. It was interesting to see the AIs repeatedly fail.
It appeared it had access to the entire book - it linked me to a website that somehow had the text. I was able to coax better responses out of it when I corrected its mistakes about plot etc.
Unless you provided it, the context window did not have the context. “Appeared to” is also not a valid way to evaluate technology.