{"id":605966,"date":"2026-09-20T06:10:13","date_gmt":"2026-09-20T06:10:13","guid":{"rendered":"https:\/\/www.newsbeep.com\/nz\/605966\/"},"modified":"2026-09-20T06:10:13","modified_gmt":"2026-09-20T06:10:13","slug":"top-labs-say-ai-may-soon-learn-to-upgrade-itself-on-its-own","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/nz\/605966\/","title":{"rendered":"Top labs say AI may soon learn to upgrade itself on its own"},"content":{"rendered":"

Once a distant ambition for technology researchers, the prospect of artificial intelligence models teaching themselves autonomously to be more efficient and capable<\/a> appears ever closer to reality.<\/p>\n

As the technology advances, developers say it is approaching \u201crecursive self-improvement\u201d, or RSI, in which AI models find ways to improve themselves and build their successor. It could bring the promise of advances in science and medicine, tech company executives say, but also risks.<\/p>\n

The uncertainty over where it all could lead is at the heart of growing fears about AI evading human control,<\/a> and possible threats to humanity, which led several AI moguls to join last weekend in a call to slow down the technology\u2019s pace of growth.<\/p>\n

Anthropic this week detailed how its model Claude<\/a> is helping the company to develop the next, more intelligent version of itself. Claude is now leading 26% of Anthropic\u2019s model research and development, which the company said means it can complete most of a given task \u201cend-to-end from a high-level prompt\u201d while still being under human supervision. The models are not working completely autonomously \u2014 at least not yet.<\/p>\n

Here are some key points about recursive self-improvement.<\/p>\n

\"Anthropic<\/p>\n

Anthropic CEO Dario Amodei. (Source: Associated Press)<\/p>\n

What is recursive self-improvement?<\/p>\n

Leading AI companies have different definitions for recursive self-improvement. Some define it as when there is any feedback from AI on model improvement, while others define it as AI working toward that goal fully autonomously.<\/p>\n

Autonomous recursive self-improvement essentially means AI that can improve itself designing the next version of the system, then the next version, and so on, said Anthony Aguirre, president and CEO of the nonprofit Future of Life Institute and a physics professor at the University of California, Santa Cruz.<\/p>\n

\u201cThe really important thing here is that as AI is doing more of it, it gets faster, because AI operates just much, much more quickly than the humans do,” he said.<\/p>\n

The fear around RSI is based largely on a runaway superintelligence emerging from that process, said John Thickstun, an assistant professor of computer science at Cornell University who studies methods that control the behaviour of AI models. But he said a more grounded view suggests a kind of recursive self-improvement has been going on in AI development for a while now.<\/p>\n

“We have already, for years, been using these models in supportive roles for creating the next version of these models. So people use the past generation of models to write code for the AI systems that then create the next generation,” he said.<\/p>\n

For years, prominent AI researchers such as OpenAI co-founder Andrej Karpathy have experimented with trying to get AI models to train and improve new AI systems. Those efforts have brought minor improvements, but not big creative leaps, Thickstun said.<\/p>\n

But AI companies today, Aguirre said, are much closer to pulling off those bigger leaps in improvement.<\/p>\n

\u201cYou can see in these plots from Anthropic over time, more and more of research is being done by the AI and it\u2019s becoming closer and closer to fully autonomous,\u201d he said. \u201cAnd the result of that success, ultimately is something that is, I think, extremely scary. I think this is probably the worst idea in the history of humanity to do this. And yes, they\u2019re doing it.\u201d<\/p>\n

\"The<\/p>\n

The OpenAI cases are the latest in a stream of out-of-control AI developments (Source: Associated Press)<\/p>\n

Some AI labs say RSI is not far off<\/p>\n

Anthropic’s recent announcement provided the public \u2014 and other labs \u2014 with some insight into RSI progress, and it encouraged its competitors to share similar metrics. Still, the company has not expressly said how close it is to achieving fully autonomous model improvement.<\/p>\n

ChatGPT maker OpenAI<\/a> announced this month that it has developed an automated \u201cresearch intern\u201d, which it defines as a system that can carry out well-defined research tasks under human direction, including \u201ctasks that would take a skilled researcher a few days.\u201d The company has said it is moving forward with the goal of creating an automated AI \u201cresearcher\u201d by March 2028.<\/p>\n

The company said in that announcement that while RSI can help align models’ actions with human values and intentions, that doesn’t mean \u201crapid RSI is necessarily an outcome we should pursue”. <\/p>\n

\u201cWhether and how to proceed must depend on our ability to preserve human control and on informed democratic choices about the benefits and risks,\u201d the company said in a blog post.<\/p>\n

Elon Musk seems more eager to forge ahead. He said in March that for xAI’s Grok models, \u201chumans are gradually getting less and less in the loop\u201d on model improvement and that \u201cevery successive model is built by the one before it\u201d, but clarified that the process was not yet fully automated. That target might be reached by the end of this year, he added, \u201cbut not later” than 2027.<\/p>\n

Microsoft and some other leading AI companies seem to be taking a different approach.<\/p>\n

Mustafa Suleyman, the CEO of Microsoft AI, has said the company is moving toward \u201chumanist superintelligence\u201d, or advanced AI capabilities that are in service of people and humanity at large. Suleyman said in a 2025 essay that this would not mean \u201can unbounded and unlimited entity with high degrees of autonomy,\u201d but rather AI that is \u201ccarefully calibrated, contextualised, within limits”.<\/p>\n

The King\u00a0warns of\u00a0the “existential dangers of such technologies falling into the wrong hands\u201d. (Source: 1News)<\/p>\n

How development slowdown talks could impact RSI<\/p>\n

A key challenge labs face \u2014 and have been facing essentially since the technology\u2019s inception \u2014 is ensuring their safety measures advance alongside the models’ capabilities.<\/p>\n

Divisions have emerged in the tech industry over calls for a coordinated AI slowdown for safety, and not every major player in the AI space has specifically commented on their path forward with RSI.<\/p>\n

Anthropic, which has been a leading voice in the calls for pacing, has said it would slow or temporarily pause its development work \u2014 assuming its global competitors also did so, and in a “verifiable manner”. <\/p>\n

OpenAI explicitly said this month it does not yet know how to \u201csafely get all the way to aligned, full RSI\u201d, adding that the company \u201ccannot assume that progress in alignment and safety will keep pace\u201d. More capable systems can become harder to monitor, it continued, but pursuing RSI is still a goal it says it values because an \u201cautomated AI researcher can also be an automated safety or alignment researcher”. <\/p>\n","protected":false},"excerpt":{"rendered":"Once a distant ambition for technology researchers, the prospect of artificial intelligence models teaching themselves autonomously to be…\n","protected":false},"author":2,"featured_media":605967,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,363,364,111,139,69,145],"class_list":["post-605966","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-new-zealand","tag-newzealand","tag-nz","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/605966","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=605966"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/605966\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/605967"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=605966"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=605966"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=605966"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}