The New York Times blocks OpenAI’s web crawler::The New York Times has officially blocked GPTBot, OpenAI’s web crawler. The outlet’s robot.txt page specifically disallows GPTBot, preventing OpenAI from scraping content from its website to train AI models.

  • BappityEnglish
    32•1 year ago

    as if a text file is going to stop them

    • SerotoninSwellsEnglish
      7•1 year ago

      NYT also uses a third party bot identification and mitigation service.