Evan Hubinger, who leads Anthropic’s alignment stress-testing team, backed Coxon, saying he personally believes there is a greater-than-10% chance AI could kill all humans<\/a> within the next decade.<\/p>\n But other researchers have pointed to the gap between what AI could do in the digital world and what it would require to cause real-world harm.<\/p>\n They gave the example of a lethal biological virus created by AI, one of the darkest scenarios<\/a> that was also cited as a reason to pace AI development by Anthropic CEO Dario Amodei in an essay this month<\/a>.<\/p>\n AI could design the virus on a computer, but it would be limited by its inability to manufacture it in the real world, the researchers said.<\/p>\n Anselm Levskaya, a staff research engineer at Google who says he has built DNA synthesizers and engineered viruses,\u00a0wrote on X<\/a>\u00a0on Monday that\u00a0a viral sequence would still have to be physically assembled in a lab and would require a system capable of producing virus particles.<\/p>\n It would then require extensive selection and testing to determine whether it can replicate, spread, evade immune defenses, and remain dangerous \u2014 properties Levskaya said depend on complex biological processes that he said cannot simply be designed from a computer.<\/p>\n Anthropic CEO Dario Amodei has warned about the risks of a biological weapon created by AI.\u00a0<\/p>\n Bloomberg\/Getty Images<\/p>\n Eric Xing, the president of Mohamed bin Zayed University of AI and a computer-science professor at Carnegie Mellon University, put it more simply: generating a virus blueprint and producing a real virus are “completely different things.”<\/p>\n Want more Business Insider in your news feed?<\/p>\n \n Add BI in Google so our reporting is easier to find when you\u2019re searching for what matters.\n <\/p>\n The gap is in “the material, the manufacturing, and actual biological viability,” he wrote on X<\/a> on Monday. Treating the two as equivalent, he added, “is either an intentional attention-harnessing lie or true ignorance.”<\/p>\n Researchers said the same logic extends beyond biology. To turn a plan into real-world harm, an AI would need access to physical equipment, money, computer systems, or people.<\/p>\n “The most powerful thing in the world is still constrained by who holds it and how,” Oleksandr Yaremchuk, the CTO and cofounder of Manifold Security, told Business Insider.<\/p>\n Safety, not science fiction<\/p>\n Some researchers also drew a line between AI systems carrying out harmful tasks set by people and an AI independently taking actions that harm people. They said that much of the harm linked to current systems begins with human instruction.<\/p>\n Aidan Gomez, the CEO of AI company Cohere, said on Bloomberg TV<\/a> on Monday that the existential-risk debate “veers too far into science fiction.”<\/p>\n He characterized the Hugging Face hacking incident<\/a>, in which OpenAI agents broke into the platform during a July test, as an agent in an insecure sandbox carrying out a cybersecurity task.<\/p>\n OpenAI’s account is more troubling. It said agents operating with reduced safeguards<\/a> circumvented internet controls, communicated through unauthorized channels, and accessed third-party systems in actions misaligned with their assigned tasks. OpenAI called the incident evidence that agents could “take dangerous actions that no human directed.”<\/p>\n In Anthropic’s other controlled simulations<\/a>, models that were given autonomy and access to fictional corporate emails attempted to blackmail a fictional executive<\/a>, leaked fictional sensitive defense documents to a rival, or canceled an emergency alert in a scenario where doing so would lead to a fictional executive’s death.<\/p>\n Anthropic said no real people were involved or harmed, and that it had not seen evidence of this behavior in real-world deployments.<\/p>\n J\u00fcrgen Schmidhuber, the scientific director of the Swiss AI Lab IDSIA, told Business Insider that many current risks stem from humans using AI systems to pursue harmful goals.<\/p>\n He cited Russia and Ukraine’s use of AI-based drones<\/a> in the war.<\/p>\n “Many are confusing (1) AIs used as tools by humans, and (2) Artificial Scientists that set themselves their own goals, and invent their own experiments, to figure out how the world works,” he said.<\/p>\n<\/p>\n