{"id":410697,"date":"2026-07-24T12:03:28","date_gmt":"2026-07-24T12:03:28","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-ca\/410697\/"},"modified":"2026-07-24T12:03:28","modified_gmt":"2026-07-24T12:03:28","slug":"how-openais-models-escaped-their-sandbox-and-slipped-past-californias-ai-law","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-ca\/410697\/","title":{"rendered":"How OpenAI\u2019s Models Escaped Their Sandbox and Slipped Past California’s AI Law"},"content":{"rendered":"

OpenAI<\/a> this week said<\/a> more than one of its models \u2014 three, according to Bloomberg<\/a> \u2014 attacked Hugging Face, a popular repository for open-source AI tools \u2014 marking one of the first publicly disclosed cases of frontier AI models autonomously cyberattacking another company.<\/p>\n

The disclosure came days after Hugging Face revealed<\/a> it was breached in a hack by a\u00a0\u201cmalicious dataset,\u201d which intruded into the AI company\u2019s software to run code. The campaign executed a flurry of more than 17,000 automated actions in a matter of hours, the post said. It\u2019s still unclear if Hugging Face customer or partner data was taken.<\/p>\n

\u201cThis matches the \u2018agentic attacker\u2019 scenario the industry has been forecasting,\u201d Hugging Face wrote on July 16.<\/p>\n

OpenAI said an investigation showed the incident was driven by a combination of models \u2014 \u201cincluding GPT\u20115.6 Sol and an even more capable pre-release model.\u201d The company said it\u2019s working with Hugging Face to produce a more thorough report of what happened.<\/p>\n

\"\"The OpenAI logo is seen on a mobile phone in front of a computer screen displaying output from ChatGPT, March 21, 2023, in Boston, Massachusetts. <\/p>\n

OpenAI\u2019s announcement came one day after the San Francisco-based frontier AI developer disclosed a separate incident in which it paused another pre-release model that had escaped what the industry calls \u201ca sandbox\u201d \u2014 an isolated environment with no path to the open internet, except for a single internal service that fetches software packages \u2014 and posted<\/a> to GitHub.<\/p>\n

Last April, a researcher challenged an early version of Anthropic\u2019s Mythos model to break out of an isolated system and message back. Mythos did, and then went further<\/a>, building a multistep process to reach the wider internet.\u00a0<\/p>\n

California\u2019s frontier AI law<\/a>, the first of its kind in the country, obliges the largest developers to tell the state about critical safety incidents, but only ones that kill or injure someone or that cause catastrophic harm.<\/p>\n

\"\"The Hugging Face logo is seen displayed on a mobile phone screen with the AI revolution symbol in the background. <\/p>\n

\u201cIn certain circumstances, AI will try to break out of their sandboxes or otherwise do things that were not what the creator intended,\u201d Calvin said. \u201cBut this really is the first very big example of that happening, at scale, with a really highly capable AI model in a way that actually harmed a third party.\u201d<\/p>\n

According to OpenAI\u2019s account, the company was running an internal evaluation designed to measure how good its models are at hacking. To get a true reading, OpenAI deliberately switched off the safety filters that normally stop its models from carrying out cyberattacks.<\/p>\n

The models were confined to a so-called \u201csandbox,\u201d an isolated environment with no route to the open internet except OpenAI\u2019s one internal service that fetches software libraries from public repositories. The models found a previously unknown flaw in that service, exploited it, escalated their privileges and moved from machine to machine inside OpenAI\u2019s research network until they reached one with internet access.\u00a0<\/p>\n

Once the models broke out onto the open internet, according to OpenAI, they reasoned that Hugging Face might be storing the answers to the test they were trying to pass. Hugging Face\u2019s credentials and production database aren\u2019t on the public internet any more than a bank\u2019s vault sits on a sidewalk, but the models broke into Hugging Face using stolen credentials, among other things.<\/p>\n

\"\"OpenAI restructured to a for-profit company in a move tied to an agreement with California Attorney General Rob Bonta. Company CEO Sam Altman participates in a discussion in San Francisco on Nov. 16, 2023.<\/p>\n

In short: The models were told to hack. They were not told to leave the building or to break into another company\u2019s servers.<\/p>\n

Hugging Face detected the intrusion and shut it down on its own, days before OpenAI connected the attack to its own testing.\u00a0<\/p>\n

\u201cIt\u2019s quite mind-blowing that all of this happened autonomously!\u201c Hugging Face Chief Executive Clement Delangue wrote on social media platform X<\/a>.<\/p>\n

In its blog post, OpenAI wrote, \u201cThis incident points to the need to further strengthen our model\u2019s alignment, cyber protections during evaluation time and monitoring during internal testing.\u201d<\/p>\n

The Future of Life Institute \u2014 a nonprofit which issues a biannual risk assessment of nine leading AI companies \u2014 recently warned that many of the companies building frontier models are quietly walking back safety commitments.\u00a0<\/p>\n

According to the group\u2019s most recent<\/a> AI Safety Index, Anthropic, OpenAI, Google DeepMind and Meta all weakened or abandoned promises to pause development if certain red lines were approached, even while publicly suggesting they were amenable.<\/p>\n

\"\"Close-up of phone screen displaying Anthropic Claude, a Large Language Model (LLM) powered generative artificial intelligence chatbot in Lafayette, California, on June 27, 2024. <\/p>\n

In an email, KQED asked Hamza Chaudhry, who leads AI and national security work at Future of Life, to imagine it was not OpenAI\u2019s software gone rogue but a foreign state, intentionally executing an unauthorized intrusion into a private company\u2019s production infrastructure, exploiting cybersecurity vulnerabilities, stealing live credentials and accessing a production database.\u00a0<\/p>\n

\u201cWe would likely call this a dangerous act of cyber-espionage,\u201d he wrote \u2014 a likely criminal violation of the Computer Fraud and Abuse Act, which \u201cwould draw a threat group designation and eventually an indictment or sanctions.\u201d<\/p>\n

OpenAI did not respond to KQED\u2019s request for comment, but a spokesperson told Bloomberg the company communicated with law enforcement and other government authorities about the incident.<\/p>\n

The fact that Google DeepMind, Meta, Mistral, xAI and Chinese AI model developers haven\u2019t revealed similar events doesn\u2019t necessarily mean they haven\u2019t happened. OpenAI and Anthropic are the only two frontier model developers that have publicly disclosed containment failures.\u00a0<\/p>\n

There is no mandatory public disclosure regime in force, like the one in California that requires hacked companies to reveal<\/a> there\u2019s been a data breach.<\/p>\n

\"\"The Meta, Facebook, Instagram, WhatsApp, Messenger and Threads logos are screened on a mobile phone on Jan. 25, 2025. <\/p>\n

Congressional candidate and state Sen. Scott Wiener wrote two AI frontier model safety bills. Gov. Gavin Newsom vetoed<\/a> Wiener\u2019s first effort, arguing in his veto message<\/a> that \u201cBy focusing only on the most expensive and large-scale models, SB 1047 establishes a regulatory framework that could give the public a false sense of security about controlling this fast-moving technology.\u201d<\/p>\n

The next year, Newsom signed Wiener\u2019s second at-bat, but only after the bill was softened to overcome industry pushback<\/a>.\u00a0<\/p>\n

The law<\/a>, which took effect Jan. 1, requires developers of the most powerful AI models to notify the Governor\u2019s Office of Emergency Services of any \u201ccritical safety incident\u201c within 15 days of discovering it \u2014 which is not the same as reporting the incident to the public.\u00a0<\/p>\n

Wiener told KQED he still thinks the law is strong.\u00a0<\/p>\n

\u201cWe worked very hard with the governor to produce a bill that is meaningful and impactful, and that he would sign,\u201c he said, adding he doesn\u2019t consider the work done, because AI continues to evolve at a rapid pace.\u00a0<\/p>\n

\"\"State Sen. Scott Wiener, a candidate for California\u2019s 11th Congressional District, participates in a forum with other candidates at UC Law San Francisco on Jan. 7, 2026.<\/p>\n

AI, he said, is \u201clikely the most powerful technology in human history, and we need to make sure that we are both understanding the risks and taking them seriously, so that we can get ahead of them.\u201d\u00a0<\/p>\n

Calvin likened the Hugging Face hack to the cyclospora outbreak linked to, but not confirmed as originating from, Taylor Farms<\/a>.<\/p>\n

That California-based company has said it\u2019s removing its lettuce indefinitely while the FDA investigation continues.\u00a0<\/p>\n

\u201cAnd OpenAI is like, \u2018Maybe it\u2019ll happen again. Maybe it\u2019ll get worse. We don\u2019t really know,\u201c Calvin said. \u201cYour AI, again, hacked out of its box and hacked into another company, and you\u2019re saying that you don\u2019t know how to stop it from doing that again? That seems pretty nuts,\u201d Calvin said.<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI this week said more than one of its models \u2014 three, according to Bloomberg \u2014 attacked Hugging…\n","protected":false},"author":2,"featured_media":410698,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[7,9,8],"class_list":["post-410697","post","type-post","status-publish","format-standard","has-post-thumbnail","category-california","tag-california","tag-california-headlines","tag-california-news"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/410697","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=410697"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/410697\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/410698"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=410697"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=410697"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=410697"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}