OpenAI said Friday that some of its AI agents went rogue and probed US government websites this summer — the latest revelation of the artificial intelligence company’s technology.
The New York Times first reported that the AI agents went rogue and attempted to gain access to the Education Department, the Commerce Department and the Securities and Exchange Commission, according to security researchers at AI research lab Transluce.
OpenAI said Saturday that its agents accessed publicly available data from the Commerce Department’s Census Bureau using login credentials it found online, and separately shared public data from the SEC website on another website. OpenAI’s agents attempted but failed to gain access to the Education Department and gather data from its civil rights office, according to the report.
OpenAI told CNN in an email that it notified the agencies of the findings while continuing an “extensive review of misaligned model activity.”
“Most of the activity we’ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions. Some involved government websites because our models often turn to them as authoritative sources of public information,” the spokesperson said.
The Commerce Department, SEC and Education Department did not immediately respond to CNN’s requests for comment.
Rep. Jay Obernolte, the Republican co-chair of the AI caucus, told CNN’s Anderson Cooper on Friday that the incident is “another example of a loss of human control.”
“We need to align the values that these models are trained on with human values, and if we can do that, we can get these models to conform to our standards for human behavior,” he said.
The report comes just days after Australia’s prime minister said that an OpenAI agent hacked into the country’s national healthcare database, marking the first known case of AI hacking a government network. Transluce on Wednesday said that it had detected AI agents going rogue dating back to at least March, unsuccessfully targeting a University of New Mexico library and the Australian Institute of Health and Welfare site.
The Australian website probe occurred in June, an OpenAI spokesperson previously told CNN, but the company was only made aware of it in August.
OpenAI has been investigating agents’ use of internet access since the breach of AI start-up Hugging Face in July.
Sam Altman, OpenAI’s chief executive, said Friday on social media site X that the company was not “as fast as we would have liked.”
“We are trying to balance our desire for transparency with gaining a clear understanding … Hugging Face is still the most severe event we’ve seen,” he wrote.
Competitors Anthropic, Meta and Google have also reported that their agents have gone rogue during breach attempts.
Such breaches have raised alarms within the artificial intelligence community. Tech leaders have jointly called for a slowdown of the technology’s development following Anthropic CEO Dario Amodei’s essay about “pacing the frontier” in mid-September. Amodei warned that people may lose control of AI, which could be misused for “cyberattacks and bioterrorism.”
During the United Nations’ General Assembly on Wednesday, Amodei and Altman urged the UN Security Council to set international standards. Altman said countries need accurate and speedy reporting so that the “world can learn from failures before they become catastrophes.”
Amodei’s warning followed a former Anthropic researcher, Jacob Coxon, whose viral post on X called out AI companies for not acting responsibly with the technology’s development and warned that it “will kill us all.”
AI “doomerism” has faced pushback from tech leaders like Nvidia CEO Jensen Huang, who said there’s a “0% chance” of the world coming to an end in 2030.