OpenAI said it has paused training of its latest artificial intelligence models as reports of AI agents going rogue mount.<\/p>\n
The decision to halt development came just hours after the company disclosed Friday that it was reviewing several incidents from the summer in which OpenAI<\/a> agents searching federal government websites acted in unexpected ways beyond what was asked of them while gathering and distributing information.<\/p>\n Separately, the AI evaluator Transluce said agents that appeared to come from OpenAI tried unsuccessfully to hack into a US Department of Education website, a detail that OpenAI has not confirmed.<\/p>\n OpenAI said in a statement that it will resume training \u201conly when we are confident that we have additional safeguards\u201d in place, adding that it expects it will have to \u201chit pause\u201d again as AI develops and other issues emerge.<\/p>\n Last week Australia\u2019s prime minister, Anthony Albanese, revealed<\/a> an OpenAI agent had breached the government\u2019s national healthcare system \u2013 but said no sensitive information<\/a> had been compromised.<\/p>\n AI labs are facing pressure from lawmakers and tech experts to slow development so they can build guardrails to stop agents from acting on their own, hacking websites and disclosing nonpublic information. The heads of both OpenAI and rival Anthropic have called for a slowdown<\/a> too.<\/p>\n It is the second time in three months that OpenAI has halted development of its models. The first came in July after disclosure of a cyber-attack targeting AI startup Hugging Face<\/a>, a now notorious incident that raised fears the industry was losing control.<\/p>\n In a meeting<\/a> with Chinese president Xi Jinping this week, Donald Trump agreed to share information on AI dangers and coordinate efforts to keep it safe. Trump believes AI fears are overblown, though, and later suggested that he plans no crackdown of his own.<\/p>\n The US is not going to be \u201cputting on brakes\u201d, Trump told reporters outside the White House. \u201cThey want to stop our progress because we\u2019re leading China by a lot, and we\u2019re going to keep it that way.\u201d<\/p>\n The latest OpenAI incidents did not appear to involve the disclosure of any nonpublic information but were concerning enough for the company to warn the federal agencies involved.<\/p>\n In the education department incident, OpenAI agents found API \u201cdeveloper keys\u201d to access government data, though ultimately only publicly available information was gathered.<\/p>\n In another case involving the securities and exchange commission, agents found information freely available to all but then posted it elsewhere on the internet, an act that went beyond what they were instructed to do.<\/p>\n US Securities and Exchange Commission spokesperson Kurt Hopfenspirger said on Saturday that \u201cno nonpublic information was accessed\u201d.<\/p>\n The Department of Education said earlier that it found \u201cno evidence of any impact to our website or databases\u201d.<\/p>\n Several other AI companies have disclosed incidents of their models going rogue and even hacking websites.<\/p>\n OpenAI\u2019s CEO, Sam Altman, said in a social media post on Friday that the Hugging Face incident \u201cis still the most severe event we\u2019ve seen\u201d.<\/p>\n OpenAI previously shared six other reports of \u201cunexpected or concerning\u201d behaviour in AI models and introduced a framework for tracking, probing and disclosing instances.<\/p>\n Get in touch<\/p>\n Contact us about this story<\/p>\n The best public interest journalism relies on first-hand accounts from people in the know. If you have something to share on this subject, you can contact us confidentially using the following methods:<\/p>\n Secure Messaging in the Guardian app<\/p>\n The Guardian app has a tool to send tips about stories. Messages are end to end encrypted and concealed within the routine activity that every Guardian mobile app performs. This prevents an observer from knowing that you are communicating with us at all, let alone what is being said.<\/p>\n If you don\u2019t already have the Guardian app, download it (iOS<\/a>\/Android<\/a>) and go to the menu. Select \u2018Secure Messaging\u2019. <\/p>\n SecureDropIf you can safely use the tor network without being observed or monitored you can send messages and documents to the Guardian via our SecureDrop platform.<\/a><\/p>\n