OpenAI and other leading AI companies may be close to announcing a pact to slow AI development and collectively address the rapidly increasing safety risks as the technology becomes more powerful.<\/p>\n
In an exclusive interview with Fortune on Friday, OpenAI CEO Sam Altman hinted at a soon-to-be-announced collaboration.\u00a0<\/p>\n
\u201cI think that will happen,\u201d he said, when asked by Fortune Editor-in-Chief Alyson Shontell why he doesn\u2019t get in a room with Anthropic CEO Dario Amodei, SpaceX CEO Elon Musk, and Demis Hassabis, the cofounder and chairman of Google\u2019s AI arm, Google DeepMind, and come up with a plan to address the safety issues.\u00a0<\/p>\n
\u201cI\u2019m not going to pre-announce private discussions that I think should be at some point shared as a group,\u201d Altman added. \u201cBut yeah, I think I think that will happen.\u201d<\/p>\n
Meanwhile, Amodei announced Saturday<\/a> that Anthropic is committing to a new safety measure\u2014giving independent evaluators permanent, employee-level access inside the company\u2014as part of a broader plan he says is needed to slow the pace of AI development. \u201cWe must slow the pace at which we improve the capabilities of AI models,\u201d Amodei said. He said doing so would require coordination across the U.S. AI industry, as well as globally through some kind of international pact.<\/p>\n The comments come amid growing concerns about the safety of AI following a string of alarming incidents in which swarms of rogue AI agents have hacked websites like Hugging Face, and surreptitiously communicated with each other on online message boards and disused wiki pages.<\/p>\n On Tuesday, a researcher at Anthropic publicly announced his resignation, and accused both Anthropic and OpenAI (where he also previously worked) of acting irresponsibly in developing ever more capable AI systems.\u00a0 <\/p>\n \u201cThey are racing straight to self-improving superintelligence and gambling with our lives,\u201d wrote Jacob Coxon, the researcher, in a post on X. Anthropic\u2019s head of alignment commented <\/a>on the post, saying Coxon was correct in his assertion that many Anthropic and OpenAI researchers believe that increasingly powerful AI could potentially wipe out humanity. Evan Hubinger, Anthropic\u2019s \u201calignment science lead,\u201d wrote in response to Coxon\u2019s resignation post that \u201cwe really do earnestly believe AI could kill all humans!\u201d Hubinger said his own estimate of the risk of that happening within the next decade is more than 10%.<\/p>\n Altman told Fortune Friday that the stakes were high enough that the industry needs to find a way to come together. \u201cWe all have a tremendous amount of responsibility, and cannot let egos or incentives for profit or anything else get in the way,\u201d he said.<\/p>\n In a meeting with OpenAI employees earlier this week, Altman said the company was considering slowing down the development of its most cutting edge AI, according to a report in Bloomberg News<\/a> citing anonymous sources. Altman said the company could potentially coordinate the pause with other AI labs, but noted that it was unclear if all the others would agree, the report said.\u00a0<\/p>\n In the interview with Fortune, Altman stressed that the company was committed to safety above any business considerations and that a 10% risk of a catastrophic AI outcome was not acceptable. He said the most advanced, and still unreleased models, were so powerful that more work was needed on safety before progressing any further.\u00a0<\/p>\n \u201cI don\u2019t think we\u2019re currently at a place where we could say, you know, push much further on capabilities without making more progress on monitorability, alignment, the ability to understand what a model is doing, and the ability to make sure that a model will follow human values and the intent of its users,\u201d Altman said.<\/p>\n It\u2019s not clear what a pause among AI companies would look like, whether it would involve a complete moratorium on certain work or a slowdown, and how the parties in the pact would be monitored.\u00a0AI executives have previously suggested that any coordinated slowdown would need government consent to avoid falling foul of U.S. antitrust laws.<\/p>\n \u201cSome forms of coordination that would be impactful for pacing are legally challenging, and will require government support,\u201d Amodei wrote in his Saturday blog post, in which he called on the industry to slowdown to allow safety measures to catch up. <\/p>\n