After striking deals with Google and OpenAI, Reddit CEO Steve Huffman is calling on Microsoft and others to pay if they want to continue scraping the site’s data.

“Without these agreements, we don’t have any say or knowledge of how our data is displayed and what it’s used for, which has put us in a position now of blocking folks who haven’t been willing to come to terms with how we’d like our data to be used or not used,” Huffman said in an interview this week. He specifically named Microsoft, Anthropic, and Perplexity for refusing to negotiate, saying it has been “a real pain in the ass to block these companies.”

Reddit has been escalating its fight against crawlers in recent months. At the beginning of July, its robots.txt file was updated to block web crawlers it doesn’t have agreements with. Then people began noticing that Reddit results were only visible in Google results — where Reddit is paid for its data to be shown — and not other search engines like Bing.

  • @jacksilver
    link
    62 months ago

    It almost does, of you browse on the phone it’s really pushy about creating an account or using the app. If they hadn’t started restricting the apps I would have never found Lemmy.