Correct, but if humans get in free and bots don’t, bots will pretend to be human. The only way it works ultimately is if every access is charged.
A year ago, Cloudflare launched pay-per-crawl, letting sites charge AI crawlers per page. Last week, it went further, announcing a pay-per-use model in which publishers get paid when their content actually appears in an AI answer, and declaring that, from 15 September, its customers’ ad-supported pages will block unpaid “mixed-use” crawlers by default.
Whether any of this works depends on whether the AI companies play along rather than route around it. But as Bruce put it to me, somebody has to try.
It doesn’t read to me that you or I as normal people looking at a website would be paying these fees.
Correct, but if humans get in free and bots don’t, bots will pretend to be human. The only way it works ultimately is if every access is charged.
Hence the market in residential proxies (often unknown to those whose residential connections are being used by AI companies for scraping)
Capitalism, uh, finds a way.