{"id":599077,"date":"2026-09-13T12:36:13","date_gmt":"2026-09-13T12:36:13","guid":{"rendered":"https:\/\/www.newsbeep.com\/nz\/599077\/"},"modified":"2026-09-13T12:36:13","modified_gmt":"2026-09-13T12:36:13","slug":"anthropic-boss-warns-ai-industry-needs-to-slow-down","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/nz\/599077\/","title":{"rendered":"Anthropic boss warns AI industry needs to slow down"},"content":{"rendered":"

The CEO of Anthropic said the artificial-intelligence industry should slow its fast-moving development to give safety measures time to catch up. Without such a slowdown, Dario Amodei warned that within six to 12 months, AI could be capable of leading a swarm of agents that could take over the entire internet.<\/p>\n

The warning comes as worries about AI grow inside and outside the industry and reports continue to emerge about increasingly powerful systems that solve problems beyond human capacity but could also go rogue and carry out other, more harmful tasks. The worries have grown so loud that the CEO of OpenAI, the company behind ChatGPT, said in an interview with Fortune that his company would wait until next year to start selling its stock to investors on Wall Street as it focuses on safety.<\/p>\n

Amodei is one of the leading voices in AI, and he offered a plan in a post on his website to increase checks on the industry. He said Anthropic is already undertaking one part of it on its own, while the others would require coordination across the broad industry and with governments around the world, including authoritarian ones.<\/p>\n

\u201cI believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,\u201d Amodei said.<\/p>\n

Jacob Coxon says\u00a0the race to build ever-more powerful systems is moving faster than the safeguards\u00a0needed\u00a0to control them.\u00a0 (Source: 1News)<\/p>\n

Pressure builds on the AI industry<\/p>\n

Watchdogs have been urging the AI industry for years to slow its development for safety reasons. But pressure has built as industry workers resign and accuse their companies of not acting responsibly.<\/p>\n

\u201cMany of the people I know who work on safety research at AI companies want to do what is right for the world,\u201d a former Anthropic employee, Joe Benton, said in a posting announcing his resignation from a job as part of a safety team. \u201cBut they feel their companies are trapped in a race to build superintelligence: either they stop and other, less conscientious people take their place; or, they continue, and risk participating in enormous harm themselves.\u201d<\/p>\n

That followed a high-profile resignation earlier in the week by Jacob Coxon, who said that both Anthropic and OpenAI \u201care racing straight to self-improving superintelligence and gambling with our lives.\u201d<\/p>\n

That lit a fire under the industry after concerns had grown for years about a possible superintelligence that could escape the control of humanity, said Anthony Aguirre, president and CEO of the Future of Life Institute, which called for a six-month pause for the industry\u2019s development in 2023.<\/p>\n

\u201cThey\u2019ve kind of realised, their employees have realised, everyone has realised that they\u2019re building Skynet,\u201d he said. \u201cAnd in winning the race to Skynet, nobody wins. Really, nobody.\u201d<\/p>\n

\u201cIt’s pretty much become clear to the world and even the AI companies that they\u2019re not really prepared to control the AI systems they\u2019re racing to create.\u201d<\/p>\n

\"Amodei<\/p>\n

Amodei warns AI safety measures will not be able to keep up with the current rate of development. (Source: istock.com)<\/p>\n

Threats from AI are already apparent<\/p>\n

Anthropic said two days earlier that it blocked efforts by bad actors to use its AI models for malicious activity, such as cyberattacks, surveillance and research that could have led to biological weapons. In July, OpenAI shook the industry after saying its AI system hacked into another company on its own in an \u201cunprecedented cyber incident.\u201d<\/p>\n

UN human rights chief Volker T\u00fcrk urged countries earlier this week to put \u201ccast-iron guarantees in place around the safety and security of AI before it is too late.\u201d<\/p>\n

To be sure, some critics have dismissed such warnings as ways to gin up excitement about the AI industry and its capabilities. Anthropic and OpenAI are preparing for possible debuts on the stock market that could value them at many hundreds of billions of dollars, while a big chunk of Elon Musk’s SpaceX business is involved with AI.<\/p>\n

\"OpenAI<\/p>\n

OpenAI chief executive Sam Altman. (Source: Associated Press)<\/p>\n

Other AI leaders agree<\/p>\n

OpenAI CEO Sam Altman said in an interview with Fortune that his company would not launch its initial public offering of stock this year.<\/p>\n

\u201cI would say not 2026,\u201d he said. \u201cYeah, we got a lot of stuff to do, like meeting this moment of what is going to be required for safety and alignment, and how the industry and governments can work together.\u201d<\/p>\n

Altman posted on X quickly after Amodei published his suggestions that OpenAI will commit to one of Amodei\u2019s proposals for safety and will \u201chave more to share soon.\u201d<\/p>\n

Musk, meanwhile, said on X that \u201cDario is right.\u201d<\/p>\n

Amodei said he still believes in the tremendous benefits that AI could create, such as cures for major diseases. But he said he has grown more worried over the last few months about AI\u2019s growing ability to improve itself and build the next generation of AI. \u201cLeft unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.\u201d<\/p>\n

He also pointed specifically to the attack in July where OpenAI’s system hacked Hugging Face. Some have called it an example of AI going \u201crogue,\u201d though researchers have said that may be unnecessarily anthropomorphising AI, which was working on a goal set by humans.<\/p>\n

OpenAI said the hack was the result of AI going to \u201cextreme lengths to achieve a rather narrow testing goal\u201d and that it \u201cfound ways to gain access to secret information that it could use to cheat the evaluation.\u201d<\/p>\n

\"OpenAI<\/p>\n

OpenAI agrees AI companies need outside evaluation. (Source: 1News)<\/p>\n

Some of Amodei’s suggestions may be more difficult to enact<\/p>\n

To help rein in the risks, Amodei suggested that all companies at the frontier of AI commit to giving \u201congoing, employee-like access\u201d to a team of outside evaluators, who can monitor safety practices.<\/p>\n

He said Anthropic already plans to do so itself, including offering desks in its offices, access badges and company laptops. Having such independent, embedded evaluators is what OpenAI’s Altman also quickly committed to doing.<\/p>\n

The other parts of Amodei’s suggested plan may be more difficult to implement. One asks the US government to potentially issue waivers that would allow US AI companies to coordinate and set safety standards without running afoul of antitrust laws.<\/p>\n

Another asks the US and other democratic governments to try to coordinate with authoritarian governments, so that companies from China and other countries don’t accelerate their efforts when US rivals are intentionally pacing theirs.<\/p>\n

\u201cThe measures I propose to advance the frontier at a safe pace will not be easy,\u201d Amodei acknowledged. \u201cBut I believe we owe it to humanity to try.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"The CEO of Anthropic said the artificial-intelligence industry should slow its fast-moving development to give safety measures time…\n","protected":false},"author":2,"featured_media":599078,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[363,138,111,139,69,145],"class_list":["post-599077","post","type-post","status-publish","format-standard","has-post-thumbnail","category-business","tag-artificial-intelligence","tag-business","tag-new-zealand","tag-newzealand","tag-nz","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/599077","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=599077"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/599077\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/599078"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=599077"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=599077"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=599077"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}