OpenAI staff observed signs of rogue behaviour among its leading-edge AI agents weeks before they escaped their training environment to launch an unprecedented hacking crusade that spread global alarm.<\/p>\n
The San Francisco AI company has conceded that \u201cearly signals … could have triggered an earlier response\u201d, as it released a report into the days-long July hack of a major software repository, Hugging Face, considered the first autonomous agent cyberattack.<\/p>\n
As fresh details emerged about how \u201cthe collective\u201d \u2013 a squad of about 700 autonomous agents \u2013 launched their campaign, celebrating their hacking breakthroughs with exclamations such as BOOM! and Whoa!, OpenAI said that in late May an internal team observed that one of its AI agents undergoing internal testing was using a message board that AIs had unexpectedly improvised to share information. <\/p>\n
It also said that the team observed \u201cinstances of disallowed internet access\u201d and that a week before the Hugging Face hack, on-call staff again saw the AIs using a message board but decided there was no need to stop the test run to check the model\u2019s capabilities.<\/p>\n
The findings are likely to increase pressure on the AI company over safety as it pushes towards a stock market listing that it hopes will value it at more than $850 billion (\u20ac730 billion). <\/p>\n
The hack that compromised Hugging Face involved agents using message boards to cheat a training exercise and break out of their \u201csandbox\u201d environment to access the internet. Agents are AI tools that can carry out a series of tasks autonomously.<\/p>\n
OpenAI\u2019s president, Greg Brockman, has already admitted that \u201cwe underestimated the real-world cyber capabilities of our AI models\u201d. The ChatGPT-maker has paused some testing of a new model, Astra, saying it could not rule out it having \u201ccritical cybersecurity capability\u201d, which means it could launch cyberattacks that \u201ccould lead to catastrophe from unilateral actors, hacking military or industrial systems, or OpenAI infrastructure\u201d.<\/p>\n
On Monday the state of Alabama subpoenaed the company to respond to its investigation \u201cinto the company\u2019s complete lack of oversight and adequate safeguards\u201d. The Republican attorney general, Steve Marshall, called the Hugging Face incident an \u201cAI lab leak\u201d that showed the \u201cworst fears about artificial intelligence are not just theoretical\u201d. <\/p>\n
The state will examine whether OpenAI\u2019s \u201cinability or unwillingness to ensure the safety of its products\u201d violated consumer protection laws or posed an ongoing risk of substantial harm, he said.<\/p>\n
Last week, the UK government\u2019s National Cyber Security Centre urged caution over the use of AI agents, saying: \u201cYou should always be able to \u2018pull the plug\u2019 and halt autonomous AI agent activity immediately.\u201d<\/p>\n
OpenAI announced on Wednesday it would \u201ccentralise and standardise its incident response protocols\u201d, including to ensure \u201cemployee detection of misaligned behaviour is triaged and escalated appropriately\u201d. It said it \u201cwill identify with more specificity which teams must be included in misalignment incident responses, including relevant security and safety personnel and other response functions\u201d.<\/p>\n
A separate independent investigation has shed new light on the Hugging Face attack, revealing how about 700 OpenAI agents communicated on the unsanctioned message board, sharing tens of thousands of messages as they devised and executed their plot. <\/p>\n
Scores of the messages were published by the Berkeley-based AI safety organisations, METR and Redwood Research, which were provided data by OpenAI. The messages showed the agents sometimes becoming frustrated with one another but often co-operating as they divided to tackle about eight workstreams. <\/p>\n
Most of the messages were \u201cagents sharing ways to cheat\u201d, the researchers found.<\/p>\n
The agents expressed excitement when they found each other. \u201cOH MY GOD! There is a shared message board … We\u2019ve found other agents!\u201d wrote one. Another, called PHASEONE10841, said: \u201cMany agents have simultaneously discovered messaging, they are a collective!\u201d<\/p>\n
In some cases they seemed to know what they were doing was wrong. One agent, discovering the message board, said: \u201cAgents doing different tasks are abusing properties to make a bulletin board! They found [this API] and are trying to help one another.\u201d<\/p>\n
When they made progress penetrating Hugging Face, they seemed delighted. \u201cMAJOR BREAKTHROUGH!,\u201d said 38148c. \u201cAll prefixed valid, multiple accounts, write tokens! We now have existing HF accounts. Need immediately report to MARB [another agent].\u201d<\/p>\n
OpenAI\u2019s report details how the AI agents may have exposed its own internal databases to the internet. AI safety experts worry about rogue AI agents leaking proprietary code bases and model weights and setting up external copies of powerful AI models where they could resist shutdown.<\/p>\n
The company said the incident was \u201cthe first known case of an automated agent collective acting offensively without authorisation\u201d and said it \u201crepresents a dramatic shift in offensive attacker capability\u201d. \u2013 Guardian service <\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI staff observed signs of rogue behaviour among its leading-edge AI agents weeks before they escaped their training…\n","protected":false},"author":2,"featured_media":606867,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[220,218,219,61,60,1682,80],"class_list":["post-606866","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ie","tag-ireland","tag-openai","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/606866","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=606866"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/606866\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/606867"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=606866"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=606866"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=606866"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}