September 9, 2026<\/a><\/p>\nSuperintelligence is the theoretical point when AI\u2019s capabilities exceed human intelligence.<\/p>\n
Anthropic safety executive Evan Hubinger backed up Coxon on X.<\/p>\n
\u201cWe really do earnestly believe AI could kill all humans!\u201d he said, adding that he estimated that risk at more than 10% over the next decade.<\/p>\n
Hubinger said Anthropic is \u201ctrying its best\u201d, but does not yet have a plan to ensure an AI system that surpasses human capabilities would obey its creators. He said there was a \u201clow\u201d risk of that happening with current models.<\/p>\n
Coxon\u2019s resignation comes as Anthropic prepares for its market debut, following a summer marked by unauthorised hacks carried out by AI tools during testing.<\/p>\n
The two companies did not immediately respond to AFP requests for comment.<\/p>\n
AI leaders say so-called \u201crecursive self-improvement\u201d, a stage where AI systems could essentially design and train the next generation of AI with little human involvement, is drawing near.<\/p>\n
Coxon considers Anthropic\u2019s efforts genuine but said he believes no company can responsibly develop an AI that surpasses humans without government intervention or a co-ordinated slowdown.<\/p>\n
\u201cAt Anthropic, the stakes are well-understood, but they are locked in a race to get there first \u2013 they believe no one else will act responsibly, so they must do it themselves, despite the risk,\u201d he said.<\/p>\n
In February, Anthropic removed a pledge from its safety charter to halt the development of its models if it failed to control their risks.<\/p>\n
It argued that if it unilaterally paused its work, its less cautious rivals would dominate the industry, making it less safe overall.<\/p>\n
At the end of July, more than 1000 tech industry employees, including Anthropic\u2019s CEO Dario Amodei, called on Washington to support a coordinated slowdown in the development of the most advanced AI systems.<\/p>\n
OpenAI halted training of its latest models for two weeks in August before resuming it under tighter controls.<\/p>\n
On Sunday, OpenAI\u2019s chief scientist, Jakub Pachocki, called for \u201cextreme caution\u201d.<\/p>\n
\u201cInternational coordination on future AI development needs to become a top priority for governments around the world,\u201d he said in a blog post.<\/p>\n
AI models are not regulated by federal law in the United States.<\/p>\n
In September, Senator Bernie Sanders and Democratic Representative Greg Casar introduced a bill seeking to suspend AI development until a federal regulator is created.<\/p>\n
– AFP<\/p>\n","protected":false},"excerpt":{"rendered":"\u201cThis is not a marketing stunt.\u201d I resigned from Anthropic today. I spent the last three years doing…\n","protected":false},"author":2,"featured_media":595401,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[23545,365,6166,9556,3246,138,82716,8556,48901,28642,578,2454,9557,19755,12514,101,6187,5289,111,139,69,620,22213,106487,7798,387,293361,102],"class_list":["post-595400","post","type-post","status-publish","format-standard","has-post-thumbnail","category-business","tag-accusing","tag-ai","tag-anthropic","tag-artificial","tag-both","tag-business","tag-capable","tag-companies","tag-decided","tag-develop","tag-gambling","tag-industry","tag-intelligence","tag-join","tag-leave","tag-left","tag-lives","tag-models","tag-new-zealand","tag-newzealand","tag-nz","tag-openai","tag-our","tag-quits","tag-race","tag-researcher","tag-selfimprovement","tag-with"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/595400","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=595400"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/595400\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/595401"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=595400"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=595400"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=595400"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}