Funnily enough, Prince says, most of the leading AI companies—who are also Cloudflare customers—welcomed his company’s intervention. “It’s because they don’t want to be suckers,” he says. “It can’t be that you pay [to license] content, but your competitors get it for free.”
Cloudflare’s blockage, Prince believes, might be the first step toward a new business model for the internet. He doesn’t quite know yet what it will look like—but he intuits that Cloudflare, as a result of its gatekeeper role, might be able to “play market-maker.” That might mean Cloudflare eventually becomes a place where websites can name their price for AI companies to train on their data—and then charge fees on those transactions. This “pay-per-crawl” model could be a substantial new source of revenue for Cloudflare’s already highly-profitable business.
It also marks a new, gloves-off era for the company. In an August blog, Cloudflare fired a shot at the AI search engine Perplexity, which it accused of scraping content even from sites that had explicitly asked to be exempt from crawlers. Perplexity denied the accusation, calling it a “sales pitch.”