Matthew Prince, the billionaire chief executive of the $80 billion cybersecurity giant Cloudflare, paints a dark picture of the internet in the not-too-distant future.
It goes something like this: artificial intelligence (AI) “answer engines” such as ChatGPT replace search as the primary entry point of the web. The shift starves an already-suffering media ecosystem of vital traffic and ad revenue, because chatbots mine the web to answer questions, but don’t then send users a list of links, and the business model that underlies the web collapses. Referral traffic plunges. Publishers large and small shut down.
“We go back to the time of the 1400s, of the Medicis, when there are five powerful families that hire all the journalists, all the academics, all the researchers. In order to do this sort of work, you have to work for one of these big five families. But they won’t be families; they’ll be the big five AI companies. There’ll be a liberal one, a conservative one, a Chinese one, probably an Indian one,” he explains. “It’d be so regressive. But is it that hard to imagine?”
Of course, the likes of ChatGPT, Claude and Google would still need the raw material — the content — that makes their machines so smart. So they divert some of their billions to hire their own journalists. “You can see Sam Altman saying, ‘Look, we need the data. Everybody’s shutting down their bureaux, so we’re just going to stand up our own version of the Associated Press.’ It would cost, like, 0.1 per cent of their GPU [chip] budget to do it. And now all of a sudden, if you want to know what’s going on in Karachi, the only place you can go get it is ChatGPT. I think that’s a really bad outcome.”
The OpenAI chief executive Sam Altman has said content creators should be paid for contributions to artificial intelligence models
SHELBY TAUBER/REUTERS
The rise of AI has sparked a bitter war between Silicon Valley and the media industry. OpenAI, which has agreed a multi-year licensing deal with News Corp, the parent company of this newspaper, has been sued by numerous artists and publishers, including The New York Times, for the alleged purloining of content without permission or compensation.
Meta, which used millions of books to train its AI models, is also being pursued in the courts. Last week, a San Francisco judge approved a $1.5 billion settlement between Anthropic and the authors of nearly 500,000 books that were illegally downloaded to train its Claude chatbot. The company will pay $3,000 per title as recompense.
And yet, a sustainable model that ensures publishers and media companies can earn enough to operate in this new world remains elusive. Prince, whose company polices traffic for about 20 per cent of the web, has a solution.
In July, Cloudflare put up digital firewalls around its clients’ sites, blocking by default AI “crawlers”, programmes that copy content. It is then up to companies to choose whether they want AI companies to pilfer their sites.
Prince’s hope is that if enough of the web says “no”, it will force the top AI companies to the negotiating table to work out a fee structure. “We have to figure out a way to develop a market for content, and the realisation that we had was, all markets require one thing: scarcity,” he explained. “The deal with Google was always that they got the content free, but then they sent you something: traffic. But these AI companies, they don’t send you anything, so they shouldn’t get the content free.”
Last week, Prince doubled down on his plan by launching a new digital currency, called NET Dollar, which would ease the way for billions of microtransactions as chatbots crawl the web and, potentially, pay every time they access a third-party site.
It is unclear whether others will join Prince’s quixotic mission. What is clear is that AI is changing the way people use the web. The Daily Mail said this year that when Google offers an AI summary at the top of its search results, click-through rates to its website are cut in half. Google disputes that its “AI Overviews” and more detailed “AI Mode” were hurting publishers, claiming that the traffic that does come from those results is “higher quality”.
Cloudflare started hearing about these shifts two years ago, not long after ChatGPT debuted in November 2022. “The media companies were coming to us saying, ‘Our businesses are getting killed by a new type of threat: AI companies’.”
What he found shocked him. Based on traffic data to the websites it services, Cloudflare found that going back ten years, for every two pages that Google crawled, it would send back one visitor. Today, tenfold: 20 pages crawled for every one visitor sent.
In other words, it is ten times harder than it used to be to get traffic. Google called “crawls to visitors” a “flawed metric.” It added: “We continue to send billions of clicks to websites every day and believe that Search’s value exchange with the web remains strong.”
Dario Amodei and Daniela Amodei, co-founders of Anthropic. The company agreed to pay $1.5 billion to a group of authors and publishers after a judge ruled it had illegally downloaded and stored millions of copyrighted books
DAVID PAUL MORRIS/BLOOMBERG VIA GETTY IMAGES
It is even more extreme with AI companies. ChatGPT scans 750 pages for every one visitor it sends to another site, Prince said, while Anthropic scans 30,000 pages per referral.
“We don’t recognise these figures,” OpenAi said. “ChatGPT search connects people with original, high-quality content from the web and makes it part of their conversation. By integrating search with a chat interface, users can engage with information in a new way, while content owners gain opportunities to reach a broader audience.”
The correlations are not necessarily causal, but Pew Research produced similar results this year, finding that users who see an AI overview click through to a link half as often as those who do not see overviews in their results.
Media armageddon is not assured. Consider music. The streaming revolution started with an explosion of piracy in 1999 when, overnight, Napster made it possible to download music and not pay for it. The resulting tumult sent the music labels into a financial tailspin. But eventually the Spotify model emerged, where every time a song is played, an artist gets paid a fraction of a cent.
For the most popular artists, the model works spectacularly. For many others, it is a ticket to penury. But, importantly, the industry overall makes more money now than it did before the Napster moment.
Prince’s hope is that the AI and media companies can strike a new bargain that serves both, not least because of his belief that as AI models equalise in terms of capabilities, they will seek to differentiate themselves by the content they offer. “In the future, it’s actually going to be the hyper local, really specific, unique content, which is going to be the stuff that is the most most rewarded,” he said.
“I’ll predict that there’ll be more money going into content creation five years from now than there is today, and we will more than replace the lost ad and subscription revenue with the revenue that comes off the back of the subscription and advertising fees from the AI companies.”