ai + ml<\/p>\n
New hardware promises greater efficiency, user experiences, and most importantly larger margins<\/p>\n
Generative AI apps and services are getting more expensive by the day as model devs grapple with surging infrastructure costs. A new generation of GPUs and AI accelerators promises relief from rising inference demand, but you won’t see the savings.<\/p>\n
After years and billions spent building bigger and better models, the great AI houses are beginning to find tangible use cases for the technology beyond chatbots and image generators.<\/p>\n
Claude Code, Codex, GitHub Copilot, and the slew of other code assistants have arguably become AI\u2019s biggest success story to date, but history tells us they won\u2019t be the last.<\/p>\n
But success is a double-edged sword. The bit barns built with borrowed money to train the Sonnets, GPTs, and Geminis at the heart of these apps and services were never meant to serve them at this scale. Inference and training are very different beasts.<\/p>\n
Those selling the shovels of the AI boom are now racing to bring new hardware better suited to serving these models. Nvidia pulled $20 billion from its war chest to acquihire<\/a> AI chip startup Groq for this very reason. And it’s not alone; everyone, from AMD and AWS to Intel and Google, is\u00a0rearchitecting<\/a> their GPUs, AI accelerators, and systems to drive down the cost per token.<\/p>\n Cheaper tokens mean better inference economics, higher margins, and the venture capitalists fanning the flames hope that OpenAI, Anthropic, and all the others might actually drag themselves out of the red one day.\u00a0<\/p>\n Your AI addiction is their opportunity<\/p>\n There\u2019s just one little problem. All that AI-optimized hardware isn\u2019t quite ready yet. Much of it is promised for the second half of this year, but it takes time to work out the kinks and ramp supply chains, which means the bulk of these new systems won’t have widespread deployments until early to mid 2027.<\/p>\n But here lies a fleeting opportunity for the flag-bearers to see how addictive their products have become, and just how much the market will bear. If Nvidia and AMD are the arms dealers of the AI age, the model devs are the drug dealers: the first hit’s free, the next ones are cheap, and before long you\u2019re hooked.<\/p>\n We\u2019re already seeing this play out. With the launch of GPT-5.5, OpenAI doubled<\/a> the price per token to $5 (input), $0.50 (cached input), and $30 (output) per million tokens. It didn\u2019t take long for Google to follow suit. The Chocolate Factory\u2019s newly-launched<\/a> Gemini Flash 3.5 is between 3x and 6x more expensive than Gemini 3.1 Flash-Lite and Gemini 3 Flash Preview.<\/p>\n These price hikes are further compounded by the fact that the agent harnesses being built atop these models are burning through tokens<\/a> orders of magnitude faster than a typical chatbot.\u00a0<\/p>\n Flat rate pricing makes a lot of sense when the majority of your customers aren\u2019t running up against usage caps. It makes a lot less sense when customers are spending $200 a month on $5,000 worth of tokens.\u00a0<\/p>\n Microsoft seems to have figured this out. It outright abandoned<\/a> seat-based pricing for GitHub Copilot and began transitioning its customers to usage-based pricing.\u00a0<\/p>\n Anthropic appears to be rethinking<\/a> its pricing model as well, but rather than moving to a pure usage-based pricing model, it\u2019s considering watering down its subscription features.<\/p>\n AI isn\u2019t the payroll paradise execs were promised<\/p>\n Executives who thought AI was going to replace a full-time employee for pennies on the dollar are in for a rude awakening. That’s not happening and it probably never will. Not when Anthropic, Google, or OpenAI can charge the equivalent of $30 an hour in tokens and make the case it\u2019s still cheaper than paying an employee $40 an hour plus benefits and unemployment insurance. Just wait, before long AI pricing will be marketed in dollars per full-time equivalent ($\/FTE) instead of dollars per million tokens.<\/p>\n AI may not be the sweet deal execs might have hoped for, but that hasn\u2019t stopped large tech firms from laying off thousands in pursuit of the technology. The FOMO has never been higher, and, if there\u2019s anything big tech loves, it\u2019s leading by example.<\/p>\n So far this month, we\u2019ve learned:<\/p>\n Meta is\u00a0laying off<\/a> about 10 percent of its global workforce, closing around 6,000 open positions, and reassigning some 7,000 workers to AI-focused divisions.\u00a0<\/p>\n Cloudflare is\u00a0cutting<\/a>\u00a0more than 1,100 workers, citing increased reliance on AI.<\/p>\n Cisco is letting about 4,000 workers go because, as its CEO Chuck Robbins put it, \u201cThe companies that will win in the AI era will be those with focus, urgency, and the discipline<\/a> to continuously shift investment toward the areas where demand and long-term value creation are strongest.\u201d<\/p>\n