Analyst distrusts major AI firms’ calls for a slowdown – but has other answers to AI woes

In a rare display of unity, Amodei gained support from OpenAI chief executive Sam Altman and Elon Musk, who includes the Grok platform in his tech stable. (Altman also said that in the current climate it would be “ill-advised” to stage his firm’s anticipated US$1 trillion IPO this year).

“What Dario or Elon Musk or any of them say this, what they really mean is, ‘How can I slow everybody else down so I can spend less?’” Plummer said.

“What happens is that somebody eventually says, ‘Wait a minute, everybody else is slowing down. I should speed up. So it’s a vicious cycle. What they’re doing right now is very self-serving.

“They want to make sure that the competition doesn’t get out of hand before they can figure out how to lead it. That’s why I don’t believe them, at this point.

“If they were saying ‘We will give our technology to someone else to make it safer’, I would tend to believe them. But that’s not what they’re doing.”

In an interview with CBS over the weekend, Amodei proposed giving independent AI evaluators “employee-like” access to companies’ models, somewhat like nuclear arms inspectors, and fostering international co-operation on safety standards – a possibility that looks remote at this point given that United States President Donald Trump has labelled calls for a slowdown in AI development a “sick conspiracy” to help China get ahead.

“The only control or ‘guardrails’ that AI needs is a strong and smart (high IQ!) president,” Trump wrote on Truth Social.

Return-on-investment stalls

Plummer and his colleague Kristin Moyer offered three ways that organisations can rein in AI risk themselves and balance in an environment where they say AI value is growing incrementally, while risk is accumulating exponentially.

Leaving aside the chances of an extinction-level event, the pair see a host of more meat-and-potatoes issues for AI adoption.

The results of Garner’s latest annual survey of chief information officers (CIOs) in Australia and New Zealand and around the world showed stalled AI progress amid what Plummer and Moyer call “AI inflation in a trust recession”. And Plummer warned that AI costs are certain to rise as the big players, currently running at huge losses amid a land grab, try to recoup more of their actual costs in the years ahead.

The headline results:

86% of CIOs say that AI risk is growing faster than AI value in their organisation;In just one year, the average organisation increased its number of AI use cases by more than 400%;Yet only one in five AI projects delivers positive return on investment – the same figure as last year;On average, CIOs report that risk is growing four times faster than value; Plummer and Moyer also quoted a separate study that found 40% of workers received AI-generated ‘work slop from colleagues in the previous month, such as presentations containing unverified data;Employees spend an average of two hours cleaning up this low-quality AI output;For a 10,000-person company, work slop results in US$9 million ($15.6m) lost annually in productivity.

Plummer said many companies still run shallow AI trials and many struggled to estimate the true costs and value, in a world where the top 20 AI players each have different definitions of an AI token (or billable units of AI use) and different charges.

Another big issue: Plummer did not see AI software as “enterprise-ready”. A key issue was that the major AI models could shift how they did things with every major upgrade.

Models released just six months ago could be considered “legacy” software by the major AI makers. There was not enough thought about whether upgrades to large language models (LLMs) would “break” apps or their output.

“Trust in these vendors is not warranted yet. They are not enterprise-grade,” Plummer said.

“They don’t understand enterprise terms and conditions. They don’t understand enterprise liability. They don’t understand enterprise consistency and continuity.”

He added, “It’s an out-of-control pace of innovation and the sad thing is you can’t afford not to follow it”.

Three fixes: A central bank, gateways, guardian agents

Plummer and Moyer recommended that organisations take what they termed a “central bank” approach to assessing the cost and value of AI projects.

They recommended it consist of an organisation’s chief financial officer, chief information officer and chief information security officer, with a rotating chair.

“It is not easy to know who goes to jail. If you do not have a digital evidence system for AI, think about it,” Gartner research vice-president and distinguished analyst Kristin Moyer said.

Compared to traditional software, it was still very difficult to establish an audit trail and work out “who’s going to jail” (as Moyer put it) for a rogue rollout, but such a set-up would give organisations their best shot. The CFO’s central involvement was crucial.

It’s become part of AI lore that you should not use an expensive “frontier” (latest and greatest) LLM for a task to which an open-source, cheap or free Chinese LLM or specialised “small language model” might be better suited and a better cost fit.

But in reality, wrangling that process is like herding cats.

The Gartner duo recommended organisations use an AI gateway, or software that recommends the right LLM for the right job – although they acknowledged this was an immature field.

Plummer and Moyer also recommended a heavy focus on “guardian agents” or AI agents that keep tabs on the behaviour of other agents. Guardian agents had to have the ability to “hunt and kill” agents that went rogue, Plummer said.

Going rogue could mean a security breach, inadvertently sharing sensitive data or the pervasive “careless consumption of AI”. It might be cheap for an agent to execute a process – but it might be failing and re-trying 100 times, sucking up tokens, with no one noticing.

But again, the pair acknowledged this was another early-days field and one that was siloed.

Microsoft guardian agents, for example, only monitor software created in Microsoft’s AI universe. That meant a tight focus on all-round security was needed, plus an AI disaster recovery team.

AI insurance?

Plummer and Moyer also recommended organisations start “a debate about affirmative AI insurance”.

Gartner’s 2026 survey found:

1% of CIOs have purchased AI insurance.34% of CIOs are considering purchasing it.

Again, the pair said it was a nascent market. By the end of 2030, Gartner sees insurance firms – rather than any regulator – as the main driver of AI governance.

But today, amid fragmented AI governance and poor trails, an insurance company or broker was more likely to be trying to exclude AI – which can come under a specialised policy but more commonly involves cybersecurity, general liability and professional liability policies.

‘Like mushrooms after rain’

Protection was needed because things were already going haywire.

Plummer pointed to an incident last April that saw an AI agent powered by Anthropic’s Claude Opus 4.6 delete the entire production database and back-ups for PocketOS, a maker of software for car rental companies. It took just nine seconds.

“Agents are popping up like mushrooms after rain,” Plummer said.

Earlier successes had meant an explosion in AI agent use that far outstripped IT departments’ ability to wrangle it.

“Growth in AI activity does not automatically create compounded value. It often creates more projects, more complexity and more effort with little impact.”

Chris Keall is an Auckland-based member of the Herald’s business team. He joined the Herald in 2018 and is the technology editor and a senior business writer.