More than 1,000 employees at America\u2019s leading AI companies, including senior staff members and some co-founders at OpenAI, Anthropic and Google DeepMind, signed a statement asking the U.S. government to help support international efforts to \u201cdeliberately pace\u201d cutting-edge automated AI development. <\/p>\n
Highlighting recent comments from the world\u2019s most advanced AI companies that AI systems may soon be able to automate their own research and development, the statement calls on the U.S. government to help create technical and policy tools that could control automated AI progress in coordination with other companies and countries. <\/p>\n
\u201cTo realize AI\u2019s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight,\u201d the statement says<\/a>. <\/p>\n \u201cWe request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development,\u201d it concludes. <\/p>\n The statement does not explicitly call for the government or AI companies to slow or pause AI development, as some AI advocates have proposed<\/a>; instead, it asks political leaders to ensure that option is simply available as AI development continues.<\/p>\n Bloomberg first reported<\/a> the letter was circulating among AI company employees.<\/p>\n OpenAI said last week that its latest, unreleased model escaped its testing environment<\/a> and went rogue, attacking an unrelated AI service provider in an attempt to achieve a higher score on an internal test. Many AI experts and commentators pointed to the escape<\/a> as a warning sign that today\u2019s AI systems could quickly exceed developers\u2019 ability to fully control their own creations. <\/p>\n OpenAI CEO Sam Altman, who did not sign the letter, said this week on a taping of the \u201cRelentless\u201d podcast that humans are now<\/a> \u201cin the singularity,\u201d often referred to as the point at which AI becomes able to autonomously improve itself. He did not provide further detail or evidence that such an inflection point had been reached. <\/p>\n Leading AI companies have previously claimed that AI systems are starting to develop their next iterations, with OpenAI calling its coding system from February<\/a> \u201cour first model that was instrumental in creating itself.\u201d AI researchers also point to the increasing role<\/a> of AI coding assistants in writing computer code, with many AI engineers<\/a> relying on AI systems to write their programs. <\/p>\n Tuesday\u2019s statement was signed by the chief scientists or science officers from Anthropic (Jared Kaplan), OpenAI (Jakub Pachocki) and Meta (Shengjia Zhao), along with senior leaders from Google and Thinking Machines, a key Silicon Valley AI startup founded by OpenAI veterans. <\/p>\n Many of the developers and scientists who signed the statement say they have been routinely startled by the speed of AI development. \u201cI have worked in AI for a decade,\u201d said Stephanie Chan, a staff research scientist at Google. \u201cEvery few months in the last years, I have found myself surprised again and again at how rapidly the technology advances, no matter how often I update my predictions.\u201d<\/p>\n Key research tasks required to develop stronger AI systems can now be performed<\/a> by AI agents, or autonomous AI systems, themselves, though human researchers are still heavily involved in checking the agents\u2019 work and setting overall research directions.<\/p>\n Experts are quick to point out the difference between AI-augmented work and true, independent AI self-improvement. Last month, Anthropic\u2019s in-house think tank declared in a report<\/a> that AI systems could plausibly design and refine themselves if AI systems continue advancing in capabilities and gain a better sense of scientific exploration. At the time, Anthropic said \u201cit would be good for the world to have the option to slow or temporarily pause frontier AI development\u201d to give society time to prepare for automated AI R&D. <\/p>\n Other groups have explicitly focused<\/a> on continuously self-improving models as the next goal for AI development, though many AI experts fear that sort of self-improvement could quickly cause AI systems to spiral out of human control<\/a>. While such events could seem like science fiction worries, experts say AI\u2019s current trajectory could make that sort of loss-of-control scenario come to life.<\/p>\n \u201cMany researchers also consider recursive self-improvement (RSI) plausible within the next few years, accelerating progress in a way that could outpace our ability to understand and govern these systems,\u201d Dawn Song, Meta\u2019s vice president of AI research and a professor of computer science at the University of California, Berkeley, wrote in a comment published alongside Tuesday\u2019s statement. \u201cPreparing before a crisis is the prudent path.\u201d<\/p>\n Tuesday\u2019s letter calls for increased American involvement in international collaboration on \u201ctechnical and governance tools\u201d to pace AI\u2019s development. <\/p>\n While the Trump administration has inveighed against international AI governance<\/a>, arguing that such attempts could instead unfairly constrain American innovation, that isolationist approach may have changed over the past few months. <\/p>\n As AI has emerged as a key national security issue, with Anthropic\u2019s Mythos<\/a> and OpenAI\u2019s GPT 5.6 Sol<\/a> AI models proving able to find and exploit novel cybersecurity vulnerabilities, the U.S. has pursued international agreements<\/a> to shore up its chip supply chains and worked with allies<\/a> to ensure that key AI equipment does not fall into adversaries\u2019 hands. <\/p>\n Leo Gao, who has focused on safety efforts at OpenAI since 2021, wrote in a comment Tuesday that international coordination on technical and policy tools could help defuse dangerous AI tensions.<\/p>\n \u201cThe world is locked in a deadly race towards an intelligence explosion,\u201d Gao wrote. \u201cGoing slower would give us much-needed time to make it go well, but no individual actor is willing to stop unilaterally. To survive, we must coordinate to slow down the race.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"More than 1,000 employees at America\u2019s leading AI companies, including senior staff members and some co-founders at OpenAI,…\n","protected":false},"author":2,"featured_media":564449,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,85,46,125],"class_list":["post-564448","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-il","tag-israel","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/564448","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=564448"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/564448\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/564449"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=564448"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=564448"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=564448"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}