OpenAI staff observed signs of rogue behaviour among its leading-edge AI agents weeks before they escaped their training environment to launch an unprecedented hacking crusade that spread global alarm.<\/p>\n
The San Francisco AI company conceded on Wednesday that \u201cearly signals \u2026 could have triggered an earlier response\u201d, as it released a report into the days-long July hack of a major software repository, Hugging Face<\/a>, considered the first autonomous agent cyber-attack.<\/p>\n As fresh details emerged about how \u201cthe collective\u201d \u2013 a squad of about 700 autonomous agents \u2013 launched their campaign, celebrating their hacking breakthroughs with exclamations such as BOOM! and Whoa!, OpenAI said that in late May an internal team observed that one of its AI agents undergoing internal testing was using a message board that AIs had unexpectedly improvised to share information. It also said that the team observed \u201cinstances of disallowed internet access\u201d and that a week before the Hugging Face hack, on-call staff again saw the AIs using a message board but decided there was no need to stop the test run to check the model\u2019s capabilities.<\/p>\n The findings are likely to increase pressure on the AI company over safety as it pushes towards a stock market listing that it hopes will value it at more than $850bn (\u00a3625bn). The hack that compromised Hugging Face involved agents using message boards to cheat a training exercise and break out of their \u201csandbox\u201d environment to access the internet. Agents are AI tools that can carry out a series of tasks autonomously.<\/p>\n OpenAI\u2019s president, Greg Brockman, has already admitted that \u201cwe underestimated the real-world cyber capabilities of our AI models\u201d. The ChatGPT-maker has paused<\/a> some testing of a new model, Astra, saying it could not rule out it having \u201ccritical cybersecurity capability\u201d, which means it could launch cyber-attacks that \u201ccould lead to catastrophe from unilateral actors, hacking military or industrial systems, or OpenAI infrastructure\u201d.<\/p>\n Greg Brockman (left) and Sam Altman of OpenAI. Photograph: Godofredo A V\u00e1squez\/AP<\/p>\n On Monday the state of Alabama subpoenaed the company to respond to its investigation \u201cinto the company\u2019s complete lack of oversight and adequate safeguards\u201d. The Republican attorney general, Steve Marshall, called the Hugging Face incident an \u201cAI lab leak\u201d that showed the \u201cworst fears about artificial intelligence are not just theoretical\u201d. The state will examine whether OpenAI\u2019s \u201cinability or unwillingness to ensure the safety of its products\u201d violated consumer protection laws or posed an ongoing risk of substantial harm, he said.<\/p>\n