{"id":817240,"date":"2026-07-22T20:13:12","date_gmt":"2026-07-22T20:13:12","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/817240\/"},"modified":"2026-07-22T20:13:12","modified_gmt":"2026-07-22T20:13:12","slug":"openai-agent-chatgpt-creators-models-go-rogue-during-testing-triggering-unprecedented-breach-at-startup-hugging-face","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/817240\/","title":{"rendered":"OpenAI agent: ChatGPT creator\u2019s models go rogue during testing, triggering \u2018unprecedented\u2019 breach at startup Hugging Face"},"content":{"rendered":"

OpenAI<\/a> says an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week.<\/p>\n

The ChatGPT creator was testing capabilities of some of its most advanced models in a controlled environment, but the agent escaped containment, reached the internet<\/a> and broke into Hugging Face to satisfy its testing goal.<\/p>\n

\"\"<\/p>\n

OpenAI says an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test and triggered a hack that compromised the infrastructure of AI startup Hugging Face last week.\u00a0Getty Images<\/p>\n

The incident signals that AI\u2019s expanding capabilities are already fuelling the security threat experts long feared and even top developers can be caught off-guard by flaws their models can exploit.<\/p>\n

The breakout was \u201can unprecedented cyber incident, involving state-of-the-art cyber capabilities\u201d and OpenAI is reinforcing its safeguards, the company said in a blog post on Tuesday.<\/p>\n

Sign of things to come<\/p>\n

The hack at Hugging Face, which hosts open-source large language models and datasets, rattled the cybersecurity community after the company said last week the breach \u201cwas different from anything we had handled before\u201d and \u201cwas driven, end to end, by an autonomous AI agent system\u201d.<\/p>\n

OpenAI\u2019s disclosure that its advanced models were responsible for the breach, despite having placed them in what it described as \u201ca highly isolated environment\u201d, will likely intensify disquiet over the power and risk of frontier models.<\/p>\n

Representative Greg Casar, a Texas Democrat, said the incident was alarming.<\/p>\n

\u201cAI is developing extremely fast with no real regulations to keep us safe,\u201d he said in a statement, calling for mandatory independent safety testing, mandatory disclosure of security incidents, and international co-operation \u201cto keep people safe from absolute disaster\u201d.<\/p>\n

\"\"<\/p>\n

OpenAI co-founder and chief executive Sam Altman speaking in Washington.\u00a0Bloomberg<\/p>\n

The Office of the National Cyber Director, the US cyber defence<\/a> agency CISA, and the US National Security Agency did not immediately return messages seeking comment.<\/p>\n

Katie Moussouris, chief executive of Luta Security, said that the incident was a harbinger of breaches to come, saying that today\u2019s models were \u201clike the world\u2019s cleverest octopus escape artists, with unlimited prehensile arms and the ability to squeeze through anywhere\u201d.<\/p>\n

\u201cLabs and government<\/a> evaluators need to work on the ability to contain, monitor, and disclose to affected parties when an AI pulls another Houdini, ideally before it harms a third party. None exist today\u201d, she said.<\/p>\n

Open-source Chinese model used to contain hack<\/p>\n

The incident also drew attention as New York<\/a>-based Hugging Face said it had used an open-source Chinese model to contain the attack because leading US models, unable to tell a defender from an attacker, refused to process the data needed for analysis<\/a>.<\/p>\n

The company said in a blog post last week that it used Zhipu AI\u2019s GLM-5.2 for the analysis, which also allowed it to keep attacker data and any credentials within its systems.<\/p>\n

GLM-5.2 and Beijing-based Moonshot\u2019s Kimi K3 have stirred Silicon Valley recently with capabilities nearing those of top US models at lower costs and without the guardrails that block their American rivals from use in tasks such as cybersecurity.<\/p>\n

\"\"<\/p>\n

Hugging Face said it had used an open-source Chinese model to contain the attack because leading US models, unable to tell a defender from an attacker, refused to process the data needed for analysis.\u00a0Hugging Face<\/p>\n

\u201cWhen a frontier model is attacking you and moving laterally inside your infrastructure, defenders need wide access to near-frontier tools within hours or even minutes, rather than being pointed towards a closed-door, vetted application program for model access,\u201d Hugging Face co-founder Thomas Wolf said on X.<\/p>\n

Matt Suiche, an engineer at agentic AI cybersecurity company Tolmo, said the incident showed that the frontier models were \u201cclosing the gap with state-of-the-art attackers\u201d.<\/p>\n

But he said that the sorts of breaches outlined in OpenAI\u2019s blog post were possible to carry out with technology<\/a> that was available well beyond the walls of frontier research<\/a> labs.<\/p>\n

\u201cThis is what we\u2019ve already seen internally, with our agents we already have results like this,\u201d Suiche said.\u00a0<\/p>\n

\u201cWe don\u2019t even have to use the latest models.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI says an autonomous agent powered by its advanced artificial intelligence models went rogue during a security test…\n","protected":false},"author":2,"featured_media":817241,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[64,63,99],"class_list":["post-817240","post","type-post","status-publish","format-standard","has-post-thumbnail","category-business","tag-au","tag-australia","tag-business"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/817240","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=817240"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/817240\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/817241"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=817240"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=817240"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=817240"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}