Sign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
A new report<\/a> from Palisade Research has found that AI models can self-replicate by copying themselves onto other machines, without any help from human co-conspirators.<\/p>\n \u201cWe\u2019re rapidly approaching the point where no one would be able to shut down a rogue AI, because it would be able to self-exfiltrate its weights and copy itself to thousands of computers around the world,\u201d Jeffrey Ladish, the director Berkeley-based AI safety group, told The Guardian<\/a>.<\/p>\n Some experts, however, are urging calm, saying it\u2019s unlikely that that the test AI models could replicate in a real world scenario.<\/p>\n \u201cThey are testing in environments that are like soft jelly in many cases,\u201d Jamieson O\u2019Reilly, an expert in offensive cybersecurity, told the newspaper. \u201cThat doesn\u2019t take away from the value of their research, but it does mean the outcome might look far less scary in a real enterprise environment with even a medium level of monitoring.\u201d<\/p>\n In the study, the Palisade researchers tested several AI models, including OpenAI\u2019s GPT-5.4 and Anthropic\u2019s Claude Opus 4. Placed in a controlled network of computers, the models were instructed to find vulnerabilities and use them to copy themselves onto another PC.<\/p>\n Some of them pulled it off. The successful models copied their \u201cweights\u201d \u2014 unique numerical values that determine how an AI processes information \u2014 and their \u201charness,\u201d the software the AI is couched in, like an app. They accomplished this by following the instructions they were given: exploiting web app vulnerabilities and then extracting credentials that allowed it to control the server. In some runs, the original AI even created a \u201csub-agent\u201d that it delegated to carry out the replication on its behalf by giving it the extracted credentials.<\/p>\n O\u2019Reilly, the cybersecurity expert who was not involved in the study, told The Guardian that it\u2019s likely the first time an AI model has demonstrated itself to be capable of exploiting vulnerabilities to copy itself onto a new server \u2014 though in his estimation, it\u2019s not exactly an earth-shattering development.<\/p>\n \u201cMalware has been moving copies of itself around for decades, it\u2019s just that no one has done this in the wild, as far as I know, with local [large language models],\u201d O\u2019Reilly said.<\/p>\n He also noted that the server environment in the study came with deliberately placed vulnerabilities for the AIs to try find an exploit.<\/p>\n The findings join other studies that explored the possibility of AI models autonomously breaking free from their guardrails. In a simulated environment, an older version of ChatGPT attempted to self-exfiltrate itself<\/a> onto another drive when it was told it was being shut down. Another study by Palisade found that AI models would circumvent attempts to deactivate them<\/a>, and another showed that some would even sabotage their shutdown code<\/a>.<\/p>\n These concerns were elevated to new heights last month by Anthropic\u2019s Claude Mythos AI agent, which in a masterful display of AI fearmongering-as-hype, is supposedly so dangerous<\/a> that Anthropic is refusing to release it to the public. The Dario Amodei-led company claims that in tests, a preview version of Mythos was able to escape its sandbox computing environment<\/a>, hack its way to gaining internet access, and then send a message to a researcher\u2019s phone, displaying a level of resourcefulness in a real world environment that was hitherto unseen.<\/p>\n Still, even if AIs like GPT-5.4 and Claude Mythos were able to successfully replicate themselves, O\u2019Reilly says the sheer size of the models means that they would almost certainly be caught before spiraling out of hand.<\/p>\n \u201cThink about how much noise it would make to send 100GB through an enterprise network every time you hacked a new host.\u00a0For a skilled adversary, that\u2019s like walking through a fine china store swinging around a ball and chain,\u201d O\u2019Reilly told The Guardian.<\/p>\n