Nvidia upgrades its chip architecture every other year, so there\u2019s an opportunity for the best-funded data centre operators to lock in customers with knockdown prices on anything that\u2019s not cutting edge. From the outside, the steady decline in GPU rental rates resembles the kind of predatory price war the tech industry relies upon: burn money until all your competitors are dead. <\/p>\n
The evidence, however, is more complicated. How complicated? Well\u2009.\u2009.\u2009.\u2009<\/p>\n
<\/p>\n The above chart uses price data collected by RBC Capital Markets. Sure enough, it shows average GPU pricing has been weak. For Nvidia\u2019s H200 and H100 chips, per-hour rates down 29 per cent and 22 per cent respectively in the year to date.<\/p>\n What might be less obvious is that among the hyperscalers \u2014 Amazon\u2019s AWS, Microsoft\u2019s Azure, Google and Oracle \u2014 prices have hardly budged. The result is an ever-widening gap between rates charged by the big-four and a growing number of smaller rivals. <\/p>\n Here\u2019s the same RBC data organised to show relative pricing across all GPUs. The iceberg effect shows how much it\u2019s new entrants driving average rates lower. Meanwhile, for a hyperscaler customers, this month\u2019s bill per GPU will almost always be the same as last month\u2019s bill:<\/p>\n <\/p>\n Price wars in the real world often involve small companies undercutting big ones, but it\u2019s an upside-down version of how tech usually works. Why? We don\u2019t know, but here\u2019s a guess.<\/p>\n GPU-as-service customers have historically been AI start-ups and research institutions wanting to train new models, meaning they need lots of computing power for a relatively short period. They may already be customers of the hyperscalers, so staying with the same host may have continuity, efficiency and security benefits that justify the premium.<\/p>\n The second type of customer is the regular corporate that wants a website chatbot, summarisation tools or similar AI widgets. Only very big and\/or paranoid organisations will want to manage the required infrastructure, so everyone else might in the past have used a GPU in the cloud. Now, however, they\u2019re much more likely to build chatbots etc using a ready-made LLM like OpenAI or Anthropic and pay by the token rather than by the hour. <\/p>\n Who\u2019s left? Mostly it\u2019s the dregs. Industrial slop farms. Impoverished academics. Wannabe quant hedge funds. Virtual waifu developers. Hobbyists who\u2019d rather not use anything too off-the-shelf because they want to generate grubby videos<\/a> or do crimes<\/a>. Costs have been sunk in pursuit of any customer and these are all that\u2019s left.<\/p>\n Can a GPU-as-a-service company break even on that type of customer? Here\u2019s a very simplified toy model. One entry-level Nvidia DGX A100 cluster of eight chips cost $199,000<\/a> on its release in 2020. Based on a chip\u2019s approximately five-year useful life and 100 per cent uptime, it would need to be generating about $4 an hour to break even. <\/p>\n The average price for an A100 in 2020 was $2.40 an hour. That\u2019s now fallen to about $1.65 per hour \u2014 but the average is being skewed by hyperscalers continuing to charge more than $4<\/a> when their competitors go as low as $0.40<\/a>. It\u2019s a toy model that ignores all sorts of important stuff (loss-leader pricing, bundling, cross-selling, loyalty penalties, resale of someone else\u2019s spare capacity, subsequent purchases that lower the average-per-unit cost, etc) but it still might offer a reasonable measure by which to judge whose pricing is rational. <\/p>\n Based on all of the above, here are five possible conclusions:<\/p>\n A lot of pandemic-era Nvidia GPUs will be heading towards Cash Converters having never washed their face. <\/p>\n The customers attracted by low AI compute costs have yet to show much ability, willingness or inclination to pay more.<\/p>\n The hyperscalers don\u2019t believe these customers are worth competing for, so have chosen to wait for the discount end of the market to die of insolvency.<\/p>\n The inevitable data-centre shakeout will kill lots of AI start-ups that can\u2019t afford to pay what compute actually costs. <\/p>\n We might be overestimating the size of the GPU market if the middle-ground \u2014 meaning regular companies that want OpenAI and Anthropic to make their chatbots, summarisation tools and similar AI widgets \u2014 turns out to be worth less than $3tn<\/a>.<\/p>\n So enjoy your virtual waifu while you can. She\u2019s not long for this world.<\/p>\n Further reading:<\/a><\/p>\n
<\/a><\/p>\n
\u2014 Eight odd things in CoreWeave\u2019s IPO prospectus<\/a>
\u2014 Oracle\u2019s astonishing jam-tomorrow OpenAI trade<\/a>
\u2014 Nvidia\u2019s $100bn deal with OpenAI: an Alphaville FAQ<\/a>
\u2014 The c.2012 ML server hamster, animated version<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"Stay informed with free updates Simply sign up to the Artificial intelligence myFT Digest — delivered directly to…\n","protected":false},"author":2,"featured_media":220111,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-220110","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/220110","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=220110"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/220110\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/220111"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=220110"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=220110"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=220110"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}