Unlike a chatbot that just answers questions, agents actually do things \u2013 book appointments, write code, manage files. And they\u2019re expensive to run, because one task can spin up dozens of agents all working at once, each racking up charges.<\/p>\n
Those charges are measured in tokens \u2013 the basic unit AI companies use to bill customers. A single agent-powered task can burn through dozens of times\u2019 more tokens than a simple chat message.<\/p>\n
Meanwhile, the computer chips and data centres needed to power all this AI can\u2019t keep up with demand, creating computing shortages and adding further uncertainty to the nascent industry.<\/p>\n
\u201cEspecially in developer circles, the cost to use AI for things like coding has grown exponentially,\u201d said Mark Barton of tech consultancy Omniux. \u201cAll the costs are really starting to skyrocket.\u201d<\/p>\n
Some companies have been so eager to use AI that they\u2019ve gone overboard in a usage binge called \u201ctokenmaxxing\u201d.<\/p>\n
\u201cIn some cases people are seeing the cost of tokens exceed the cost of the employee within a month or two of use, just because they\u2019re using it too much,\u201d says analyst Jack Gold of J.Gold Associates.<\/p>\n
Smarter spending<\/p>\n
Even Meta \u2013 which earlier this year encouraged employees to use as many tokens as possible as a measure of productivity \u2013 has had second thoughts.<\/p>\n
\u201cNobody should be using AI tools just for the sake of using them,\u201d chief technology officer Andrew Bosworth wrote in a memo to staff, reported by the Wall Street Journal.<\/p>\n
Uber\u2019s chief operating officer this week went a step further, raising eyebrows by saying all this AI spending was showing no noticeable increase in productivity.<\/p>\n
To cut costs, some companies are switching to free, open-source AI models that anyone can download \u2013 not as powerful as ChatGPT or Anthropic\u2019s Claude, but good enough for many tasks.<\/p>\n
Others are moving to smaller, more specialised models built for specific industries like real estate or finance, rather than giant general-purpose ones.<\/p>\n
And some are simply breaking big AI tasks into smaller steps, handing each piece to the cheapest model that can handle it.<\/p>\n
The price difference can be dramatic.<\/p>\n
\u201cThe big large monolithic model, it\u2019s $15 per million tokens, but you can get that down to like five cents if you use the smaller mini model,\u201d Adrian Balfour, of consultancy Envorso, said.<\/p>\n
All of this points to AI becoming more like a commodity \u2013 where the specific model matters less than finding the right one at the right price.<\/p>\n
But don\u2019t count out the big players and their state-of-the-art models just yet.<\/p>\n
\u201cThe most advanced users\u201d will always be willing to pay for the best, John Belton, a portfolio manager at Gabelli Funds, said.<\/p>\n
\u201cIt\u2019s a growing pie.\u201d<\/p>\n
– AFP<\/p>\n","protected":false},"excerpt":{"rendered":"Unlike a chatbot that just answers questions, agents actually do things \u2013 book appointments, write code, manage files.…\n","protected":false},"author":2,"featured_media":457768,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[74,365,9556,363,364,1975,232544,26695,156155,8556,127489,39923,14739,8455,9557,111,139,69,87100,42370,1355,145,223,165],"class_list":["post-457767","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-after","tag-ai","tag-artificial","tag-artificial-intelligence","tag-artificialintelligence","tag-at","tag-baulk","tag-bills","tag-binge","tag-companies","tag-disruptive","tag-embrace","tag-expensive","tag-getting","tag-intelligence","tag-new-zealand","tag-newzealand","tag-nz","tag-rethink","tag-soaring","tag-starting","tag-technology","tag-the","tag-their"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/457767","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=457767"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/457767\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/457768"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=457767"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=457767"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=457767"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}