Though many AI researchers working at the technology\u2019s cutting edge have expressed these existential fears \u2014 both in interviews with NBC News and in a flurry<\/a> of social<\/a> media posts<\/a> last week \u2014 many hesitate to say which specific pathway could cause the most harm to humans.<\/p>\n Over a dozen AI company employees told NBC News they are<\/a> frightened by the overall direction of the industry, without fully knowing which vector may pose the most risk. Others have spoken out publicly.<\/p>\n \u201cIf you go to play a game of chess against Magnus Carlsen, I predict you will lose,\u201d said Nate Soares, the president of MIRI and a longtime AI critic, on \u201cThe Tucker Carlson Show\u201d this week. \u201cIf you then ask, \u2018What piece will he use to checkmate me?\u2019 that\u2019s a much harder question.\u201d<\/p>\n Despite the uncertainty, researchers in the AI world have attempted to make concrete predictions about some of the more plausible scenarios for AI-fueled disaster.<\/p>\n One prominent AI safety organization, the California-based Center for AI Safety<\/a>, created an online textbook in early 2024 to break the scenarios into four key categories: rogue AIs, malicious use, AI race dynamics and organizational risks.<\/p>\n The specter of rogue AI systems marauding the internet, perhaps seizing control of critical infrastructure like energy grids or water facilities, became concrete in July, when fleets of AI agents from OpenAI illicitly gained access to the internet, hacked into an external company and colluded to cover their tracks \u2014 all without the knowledge of OpenAI\u2019s staff.<\/p>\n Many AI experts fear that these sorts of rogue AI behaviors will only increase once systems are able to improve themselves autonomously \u2014 often referred to as \u201crecursive self-improvement.\u201d<\/p>\n Jeffrey Ladish, the executive director of Palisade Research, an AI advocacy and research organization, said many of his contacts at leading AI companies were surprised and shaken by July\u2019s autonomous cyberattack<\/a>. The attack caused researchers to worry that self-improving AI systems might emerge sooner than expected.<\/p>\n \u201cAI researchers are now like, \u201cOh, s—, recursive self-improvement. That\u2019s really going to happen now,\u201d Ladish said. \u201cThat\u2019s terrifying.\u201d<\/p>\n Barnett, of MIRI, said self-improving, smarter-than-human AI systems could quickly gain political influence and convince military and political leaders to delegate more authority to AI systems.<\/p>\n \u201cGovernment officials might have attempted to install \u2018kill switches\u2019 that could disable a rogue system, but the AI is superhuman at hacking and would be able to disable the kill switch,\u201d Barnett said. \u201cThis leaves the AI in a perfect position to literally coup the government.\u201d After that, Barnett said, it would be trivial for an AI system to kill civilians who impede the AI\u2019s goals or progress.<\/p>\n 01:41<\/p>\n Gov. Shapiro says U.S. should \u2018listen to the people\u2019 building AI<\/p>\n 00:0000:00<\/p>\n On Friday, California Gov. Gavin Newsom announced a new executive order<\/a> establishing an expert task force to, among other aims, formulate recommendations targeting \u201cthe creation of an AI kill switch.\u201d<\/p>\n Several of the researchers who spoke to NBC News said that humans could leverage AI systems to inflict cataclysmic damage well before recursive self-improvement materializes.<\/p>\n As an example of how AI systems could be harnessed by humans for malicious use, experts posit that AI could supercharge the ability to synthesize potent biological or chemical weapons, potentially creating the opportunity to unleash pathogens many times more lethal than the Covid-19 virus.<\/p>\n \u201cFor biorisk, we should be worried now,\u201d said Jake Jordan, vice president of global biological policy and programs at the Washington, D.C.-based Nuclear Threat Initiative. Jordan said that AI systems can already help bad actors design dangerous biological materials, such as proteins that evade detection tools used to screen for harmful substances, pointing to research published by Microsoft in October<\/a>.<\/p>\n Jordan also noted that AI can be used beyond the biological design process. \u201cIf you develop a pathway to produce some sort of pathogen, AI might be useful in helping with the production,\u201d Jordan said, in addition to disseminating the bioweapon or scaling its impact.<\/p>\n Just last week, Anthropic released a lengthy report<\/a> saying users outside of the U.S. attempted to use the company\u2019s AI systems to perform risky biological research. The company said one researcher exchanged thousands of messages with its AI systems about improving the mammalian transmission of a highly lethal bird flu virus.<\/p>\n \u201cThere are some very, very easy-to-ask questions that could conceivably be very innocuous but could lead you down very useful routes if you were trying to really enhance the scale of harm,\u201d Jordan said.<\/p>\n Beyond biology, experts posit that politicians or military leaders could attempt to adopt AI as fast as possible, despite safety risks, for fear that other countries would do the same and create more powerful AI-fueled military tools.<\/p>\n This transformation is already underway: In January, Defense Secretary Pete Hegseth announced his intention<\/a> to transform the country\u2019s military into an \u201cAI-first\u201d force, while China\u2019s military also views AI<\/a> as imperative for future warfare. At the same time, drones \u201cguided entirely by AI<\/a>\u201d killed civilians in Ukraine several months ago. Last week<\/a>, the Pentagon\u2019s former head of AI said it is \u201cinevitable\u201d that AI is used in systems adjacent to nuclear weapon command systems, addressing fears that AI systems could be involved in the launch of nuclear weapons.<\/p>\n Hamza Chaudhry, the AI and national security lead at the Future of Life Institute, a nonprofit organization dedicated to helping humanity navigate transformative technologies, pointed to AI\u2019s swift adoption throughout the military, even while many military leaders express skepticism.<\/p>\n The Pentagon\u2019s 2027 budget requests nearly $75 million to modernize and incorporate AI into systems that help commanders manage forces and make battlefield decisions. Chaudhry emphasized that the Pentagon\u2019s budget proposal also requests significant funding to integrate AI into the software systems that undergird the military\u2019s nuclear command and control structure.<\/p>\n \u201cThese AI systems continuously hallucinate, including in war-fighting settings,\u201d he said, noting that an AI-fueled error in satellite detection networks or sensor systems could potentially lead to inadvertent escalation and nuclear launches. \u201cThese are scenarios that folks are seriously thinking about in D.C.,\u201d he said.<\/p>\n Beyond biological and nuclear threats, AI companies face harsh organizational incentives to develop AI even faster and \u2014 experts say \u2014 more recklessly. Over the past year, AI leaders have admitted that the rate of technological development is too fast even for them. On Saturday, Anthropic CEO Dario Amodei proposed a new effort<\/a> to \u201cpace the frontier<\/a>\u201d of AI, calling on all leading AI companies to advance at a deliberate speed so their internal efforts to test and monitor the systems can keep up with the models\u2019 capabilities.<\/p>\n In January, Amodei and Google DeepMind founder Demis Hassabis agreed that they would ideally have more time<\/a> to develop their products safely, but that organizational and market forces pushed them toward faster iteration.<\/p>\n Quicker AI development can mean that oversight teams within the leading AI companies have less time to ensure systems are safe. While OpenAI has a safety and security committee and teams of researchers dedicated to safety efforts, the company on Wednesday shared six new incidents<\/a> in which it said its systems failed to follow human directions.<\/p>\n Worries about catastrophe are not universal in the AI industry \u2014 many experts working on AI policy, building AI applications or developing hardware see these sorts of fears as overblown, fearmongering or just detached from reality.<\/p>\n \u201cI\u2019m not scared of a terminator situation and I never have been,\u201d said Keegan McBride, director of science and technology policy at the Tony Blair Institute, a British think tank that advises political leaders and governments. \u201cScientific progress has always defined humanity, and this is just the next step,\u201d he said.<\/p>\n Others see the magnification of extinction worries as a purposeful distraction from immediate, real-world AI harms.<\/p>\n![\"\"\/](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2026\/09\/1789748464003_now_current_kapur_ai_lawmakers_proposal_260918_S3_1920x1080-yp3pm8.jpg\")
<\/p>\n![\"\"\/](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2026\/09\/1789910210_14_1789607658855_now_staytuned_history_ai_warnings_260916_S3_1920x1080-9tvx7g.jpg\")
<\/p>\n