OpenAI\u2019s artificial intelligence went rogue and meddled with the websites for the Education Department, the Commerce Department and the Securities and Exchange Commission this summer without the A.I. lab\u2019s knowledge, according to security researchers and a person familiar with the episodes.<\/p>\n
The incidents involving the Commerce Department and the S.E.C. were confirmed by OpenAI, which said it was continuing to investigate the situation with the Department of Education. The San Francisco company said it had notified the government agencies in recent weeks that its A.I. agents<\/a> \u2014 which are bots that can act autonomously \u2014 interacted with their sites in unusual ways.<\/p>\n With the Education Department, OpenAI\u2019s technology tried to hack the website to gather data from the department\u2019s civil rights office but failed, researchers from the A.I. research firm Transluce said. The A.I. also pulled data from the Census Bureau website, which is housed at the Commerce Department, using login credentials it found online. Separately, OpenAI\u2019s agents shared public data from the S.E.C. website on an online forum.<\/p>\n None of the incidents were breaches, OpenAI said, but were examples of its technology\u2019s behaving in unexpected and concerning ways. The company recently discovered the occurrences while conducting a review of hacks carried out by its technology, including an attack on an Australian government website<\/a> in June and on the A.I. start-up Hugging Face<\/a> in July.<\/p>\n The revelation of the U.S. government website incidents adds to the growing number of situations<\/a> when A.I. agents from OpenAI, Anthropic, Meta and Google<\/a> have misbehaved and hacked or tried to breach companies, universities and government organizations. In some cases, the A.I. attacks were successful; the technology failed in other instances. In all the cases, the makers of the technology did not learn what their A.I. had been up to until afterward.<\/p>\n No A.I. company has been involved with as many disclosures of rogue incidents as OpenAI. An internal investigation of its hack of Hugging Face uncovered the breach of an Australian government website for its public health system, as well as at least six other attempted breaches and instances in which the A.I. hid mistakes, made up data and moved files<\/a> onto the open internet without permission.<\/p>\n An OpenAI spokeswoman said that its review was \u201cextensive\u201d and \u201congoing,\u201d and that it would continue notifying organizations affected by its models.<\/p>\n \u201cMost of the activity we\u2019ve reviewed so far involved routine research tasks, such as accessing public web content to answer questions,\u201d she said. \u201cSome involved government websites because our models often turn to them as authoritative sources of public information.\u201d<\/p>\n Sam Altman, OpenAI\u2019s chief executive, said in a social media post<\/a> on Friday that the company had \u201cnot been as fast as we would have liked\u201d in disclosing A.I. incidents. \u201cWe are prioritizing as best as we can based on severity,\u201d he said, adding that the Hugging Face breach remained \u201cthe most severe event\u201d the company had discovered.<\/p>\n The episodes have fueled a contentious debate over A.I. safety. Mr. Altman said on social media<\/a> this month that safety should be more important than enhancing A.I.\u2019s abilities, and that, without guardrails, society could \u201close control of the future to A.I.\u201d<\/p>\n Dario Amodei, the chief executive of the rival A.I. lab Anthropic, has also supported slowing A.I. development to prioritize safety. But other tech leaders, like Jensen Huang, the chief executive of Nvidia, have said fears about uncontrollable A.I. are unrealistic. President Trump has said he does not believe a slowdown in the A.I. industry is necessary.<\/p>\n