The AI mania that\u2019s spread across Silicon Valley like a fever over the past few years is running up against some hard economic realities.<\/p>\n
In recent weeks, big tech companies have been forced to admit that spending on tokens\u2014the basic unit of measurement for AI usage\u2014has gotten out of control. Amazon had to shut down an in-house competition to use as many tokens as possible at work, telling employees, \u201cPlease don\u2019t use AI just for the sake of using AI,\u201d according to Business Insider<\/a>; Uber has reportedly<\/a> capped employee spending on tokens to $1,500 per month after the company exhausted its annual AI budget earlier this year. And most tellingly, the companies building the big AI models have also woken up to this sobering reality. At a recent event<\/a> hosted by OpenAI, company chief executive Sam Altman admitted that token usage had become \u201ca huge issue\u201d for companies that were promised big productivity gains if they incorporated AI across their organization.<\/p>\n That\u2019s a hard pivot from just a few months ago, where the general vibe across the industry was the more that employees use AI, the better off they\u2014and the companies they work for\u2014will be. So-called \u201ctokenmaxxing\u201d became a meme, and more or less synonymous with \u201cfuture-proofing\u201d: in a day and age when everyone and their neighbors are using AI, those who know how to use AI will have a sharp edge. Not every job will necessarily be replaced by AI (so the thinking goes), but employees who don\u2019t use AI will definitely be replaced by those who do.<\/p>\n But AI has always been expensive, and training and inference costs for new models are only getting higher<\/a>. Meanwhile, the industry\u2019s dedicated push into agents\u2014AI systems that can work with little to no human oversight for extended periods of time\u2014has led to a token usage explosion. One preprint study<\/a> posted in April found that agents use 1,000 times as many tokens as other AI systems.\u00a0<\/p>\n It\u2019s the companies and individual users who have overwhelmingly had to eat those costs. No wonder some developers have resorted to pirating free online chatbots<\/a> like Chipotle\u2019s customer service bot, Pepper, to bypass the big companies\u2019 token-hungry models. GitHub announced<\/a> earlier this week that it was rolling out a new payment model in which users would be charged by the number of tokens they burn. Judging from some of the early user feedback<\/a>, it hasn\u2019t been going well. <\/p>\n Big tech desperately needs to find a new way to sell people on the future of AI without the exorbitant token costs. If they don\u2019t, companies and users will just switch to some open model they can use for free.<\/p>\n Close to the edge <\/p>\n Some big tech companies have literally been forced to the edge by the rising costs of AI usage.\u00a0<\/p>\n Microsoft<\/a> and Google<\/a> recently announced new AI products\u2014Gemma 4 12B and the RTX Spark laptop, respectively\u2014that are based on so-called \u201cedge\u201d computing. That\u2019s when a model is powered by computing power from a specific device, rather than by the cloud (i.e., energy-guzzling data centers). Obviously, a model of the magnitude of a Claude Opus 4.8 or a GPT-5 isn\u2019t going to be able to be run directly from your laptop; that\u2019s like trying to provide enough energy for a Falcon 9 rocket launch by plugging a stationary bike into a generator. But the logic behind Microsoft and Google\u2019s new products is that actually, not everyone needs the latest, greatest, most token-hungry models directly in the devices they\u2019re using daily. For most people most of the time, a smaller, leaner model will work just fine. And crucially, it will save everyone some money on tokens.<\/p>\n Make no mistake, Microsoft\u2019s and Google\u2019s investments in edge computing are minuscule compared to what they\u2019re spending on data centers; cloud computing is still very much the backbone of both of their business models. But in their embrace of edge computing, we\u2019re seeing an, at least tacit, acknowledgement that the cost of massive AI models just isn\u2019t worth the squeeze that it\u2019s placing on most consumers.<\/p>\n Water promises <\/p>\n While they\u2019re pushing new edge computing products\u2014promising powerful AI capabilities at a lower cost\u2014Microsoft and Google are also trying to pacify a public that\u2019s become increasingly concerned over data centers\u2019 water demands. (Data centers usually use water to keep GPU clusters from overheating.) On Tuesday, during the opening keynote of Microsoft Build, the company\u2019s annual developer conference, CEO Satya Nadella claimed Microsoft\u2019s new data centers\u2019 annual water usage \u201cis roughly equivalent to what a single restaurant would use.\u201d\u00a0<\/p>\n