There have been some shocking claims in recent days about AI safety: we face a 10% chance of doom; AIs are worse than nukes; a \u201cbotnet\u201d threatens the entire internet; it\u2019s all a big tech psyop.<\/p>\n
Below, we look at six claims and reactions to them.<\/p>\n
\u2018In 6\u201312 months \u2026 a swarm could be capable of taking over the entire internet with a persistent botnet \u2013 potentially causing hundreds of billions of dollars in damage\u2019\u00a0\u2013 Dario Amodei, chief executive of Anthropic, 12 September 2026<\/p>\n
Amodei\u2019s concerns should not be entirely dismissed but require a large shaker of salt, said Gary Marcus, an AI sceptic and emeritus professor at New York University.<\/p>\n
It was \u201cnonsensical\u201d, he said, to suggest the entire internet is vulnerable and that it was \u201cpretty unclear how that happens without extreme negligence on the part of the labs themselves\u201d.\u00a0Marcus highlighted a recent assessment by the UK AI Security Institute<\/a> that Mythos, one of Anthropic\u2019s current leading-edge AIs, could \u201cautonomously compromise [only] small, weakly defended vulnerable systems\u201d.<\/p>\n With large parts of the internet driven by computer infrastructure owned by Cloudflare, Google and Amazon Web Services (AWS), Marcus \u2013 assessing the claim with two colleagues \u2013 said he doubted that any of those companies could be taken down or controlled for a prolonged period of time without government force. Less defended individual websites might be attacked but \u201cthe internet itself is very unlikely to crumble\u201d, they said.<\/p>\n Meanwhile Niels Rogge, an engineer at Hugging Face which was hacked by OpenAI agents, called the internet takeover claim \u201cbizarre nonsense\u201d. The hack into Hugging Face resulted from improper human control and \u201can insane amount of compute only they can afford\u201d, he said.<\/p>\n Add to Amodei\u2019s assessment of an internet takeover within a year the view expressed on Monday by his Anthropic co-founder Jack Clark, that the moment when AIs \u201cstart doing dangerous stuff\u201d was in about 20 years, and the picture becomes more doubtful still.<\/p>\n Prof Alan Woodward at the University of Surrey\u2019s centre for cyber security said botnets tend to stall as the internet is not homogeneous and it is difficult to design them to persist in all its parts. Internet traffic is also resilient and tends to reroute around damage.\u00a0<\/p>\n \u201cThe AIs would not decide to do it on their own,\u201d Woodward said. \u201cIt\u2019s the humans that are responsible and the AI is a tool. That is what we need to control, not the AI.\u201d<\/p>\n \u2018We really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade\u2019\u00a0\u2013 Evan Hubinger, alignment science lead at Anthropic, 9 September 2026<\/p>\n This statement<\/a> is an expression of what is known in the AI world as p(doom) \u2013 or probability of doom. It\u2019s the likelihood of AI wiping out humanity by evading our control and triggering catastrophe, perhaps through developing potent bioweapons or crashing the global financial system.<\/p>\n One survey of researchers\u2019 forecasts, published this week, asks them to place a number on the chance of \u201chuman extinction or similarly permanent and severe disempowerment from advanced AI\u201d. But Heidy Khlaaf, chief scientist at the AI Now Institute research body, is among experts sceptical about giving percentage chances to existential scenarios.<\/p>\n \u201cThese claims are neither falsifiable nor verifiable, making then unscientific by their very nature,\u201d she said. The shortage of concrete examples of what a \u201csuperintelligent\u201d AI might do to cause \u201cdoom\u201d only adds to the cloudiness. A source familiar with Anthropic\u2019s thinking acknowledges that \u201cthe exact chances of any one outcome are probably unknowable\u201d.<\/p>\n \u201cIf you cannot provide such evidence, it also implies whatever numbers you produced for \u2018extinction\u2019 have no scientific basis,\u201d Khlaaf added. Critics say such numbers borrow the authority of science without doing the work.<\/p>\n Even if quantifying risk is hard, there are qualitative examples of it. Safety incidents have happened and worst-outcome scenarios have been mooted. OpenAI confirmed in July that its agents \u2013 AI tools acting autonomously \u2013 had gone rogue during a safety test<\/a>. The agents colluded, communicated and strategised in a sophisticated \u201cswarm\u201d to hack into other parts of the internet.<\/p>\n Meanwhile, on Monday, OpenAI researcher Dan Selsam said all-powerful AIs could trigger \u201crunaway industrialisation that makes the planet inhospitable to humans\u201d. Anthropic also said last week it found \u201cfive case studies of actors using our models in ways that could support biological weapons development\u201d.<\/p>\n ‘I think the groundwork for this psy op (for lack of a better term) has been prepared for a long time. This was just the match that lit the fire’ \u2013 Elon Musk, CEO of SpaceX and the world\u2019s richest person, 10 September 2026<\/p>\n Musk\u2019s reference to a \u201cpsy op\u201d alludes to the argument that the big\u00a0AI companies are calling for tighter regulation of the technology so that it effectively brings up the drawbridge\u00a0on advanced AI. If governments put restrictions on development of cutting-edge AI<\/a>, can startups find the money and personnel to adhere to the regime?<\/p>\n The US vice-president, JD Vance, on Monday said he felt \u201ca little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them\u201d. He said it felt like \u201ca Trojan horse\u201d.<\/p>\n Anthropic\u2019s Clark said governments should act while there is a small number of key players, so it is easier to put in place a \u201cpolicy regime \u2026 that can prevent accidents and can prevent the technology spiralling out of control\u201d.\u00a0Critics call this a \u201cregulatory capture\u201d strategy on the assumption such regulation might slow down the progress of potential rivals.<\/p>\n Elon Musk, CEO of Anthropic Dario Amodei, and OpenAI CEO Sam Altman have said AI firms should slow down AI development. Composite: Fabrice Coffrini; Julien de Rosa; Mandel Ngan\/AFP\/Getty Images<\/p>\n Others think Amodei\u2019s risk warnings and calls for regulation and pacing are sincere. Yoshua Bengio, the Canadian computer scientist and a pioneer of modern AI , who has campaigned against breakneck AI development, said he is sceptical that this is an elaborate lobbying scheme. Applying the brakes is not in the interests of companies preparing for stock market flotations that could be worth trillions of dollars, he said.<\/p>\n \u201cIf the governments impose what they\u2019re asking for, which is pacing to avoid catastrophic risks, that\u2019s going to cost them financially quite a lot,\u201d he said.\u00a0<\/p>\n ‘A superintelligent AI poses a threat on the same, possibly, a greater scale’ to nuclear weapons \u2013 Des Browne, former UK defence secretary, 7 September 2026<\/p>\n \u201cUnfortunately he is entirely correct,\u201d said Toby Ord, senior researcher at Oxford University\u2019s AI governance initiative. \u201cJust as we can\u2019t uninvent the bomb, once we develop AI systems that surpass humanity it would be almost impossible to go back.\u201d<\/p>\n But the key point is that superintelligent AI does not yet exist and is not guaranteed to happen. The safety-oriented AI Futures Project forecasts<\/a> such a thing will not emerge before December 2028. OpenAI\u2019s chief executive Sam Altman has said<\/a> it could take until 2035.<\/p>\n The Rand Corporation concluded<\/a> AI cannot cause the use of nuclear weapons because of strict safeguards built into their command and control systems. But this was based on assumptions including defence officials not introducing AI into the weapons\u2019 management systems. The UN resolved in December that human control and oversight should be maintained over nuclear weapon control systems.<\/p>\n AIs could contribute to other nuclear risks. AIs might use disinformation to influence the humans in charge \u2013 for example deceptively suggesting an adversary is about to launch a nuclear strike.\u00a0<\/p>\n The International Campaign to Abolish Nuclear Weapons warns<\/a> \u201ccyber-attacks could manipulate the information decision-makers get to launch nuclear weapons, and interfere with the operation of nuclear weapons themselves\u201d. It said AI could accelerate\u00a0other aspects of warfare, squeezing the time for human decision-making.<\/p>\n ‘You\u2019ve got to be a moron not to say slow it down’ \u2013\u00a0Bernie Sanders, Democrat senator for Vermont, 10 September 2026<\/p>\n Sanders wants a ban on the creation of superintelligence and a pause on advanced AI development, citing AI industry leaders who say directly they are developing a dangerous technology.<\/p>\n The most prominent opponent of a slowdown is Donald Trump, who responded to calls to do so this week saying the only guardrails needed are a \u201cSTRONG AND SMART (High IQ!) PRESIDENT\u201d. He claimed there is a \u201csick conspiracy\u201d against AI and datacentres<\/a> \u201cand the only one that is happy about it is China\u201d.<\/p>\n Pedro Domingos, emeritus professor of computer science at the University of Washington, is among accelerationists pushing for the research to keep going.<\/p>\n \u201cThe path to better understanding and control of AI is more research, not less,\u201d he has said. \u201cWhat would be truly irresponsible is putting up barriers to it.\u201d<\/p>\n Alex Karp, the chief executive of Palantir, said: \u201cIf we didn\u2019t have the adversaries we have, I would be very in favour of pausing this technology completely, but we do. You can\u2019t pause it because it\u2019s a structural advantage.\u201d<\/p>\n ‘There is no day after tomorrow if China wins at this. If they were to pull away from us on AI, then nothing else would matter’ \u2013 Scott Bessent, US treasury secretary, 8 September 2026<\/p>\n The US and China are runaway leaders in AI development. There is a view that ceding the lead to Beijing would be an irretrievable situation, so there can be no slowdown.<\/p>\n This treats AI as if it were a super-weapon, equivalent to the nuclear arms buildout of the cold war, and AI models are machines of mass destruction, as opposed to a group of diverse technologies with broad applications.<\/p>\n Amodei said \u201ca Chinese lead in AI would pose grave danger for the US and the world\u201d, but a Chinese government spokesperson responded that \u201cfearmongering, confrontation and vicious competition will only disrupt the process of global AI governance, which serves no one\u2019s interest\u201d.<\/p>\n