{"id":900527,"date":"2026-09-21T21:24:10","date_gmt":"2026-09-21T21:24:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/900527\/"},"modified":"2026-09-21T21:24:10","modified_gmt":"2026-09-21T21:24:10","slug":"almost-started-a-war-report-reveals-dark-reality-emerging-as-ai-permeates-civilisation","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/900527\/","title":{"rendered":"\u2018Almost started a war\u2019: Report reveals dark reality emerging as AI permeates civilisation"},"content":{"rendered":"<p>Yep, we\u2019re here already.<\/p>\n<p>Over the weekend, the US military reportedly rushed aircraft into the air and held armed troops on standby to board a Chinese ship.<\/p>\n<p>A chatbot had falsely linked its cargo to a nuclear weapons program.<\/p>\n<p>The operation was stopped, but only after officials bothered to take a closer look at the intelligence report that influenced some of the most powerful decision makers on the planet.<\/p>\n<p>One source described the report as \u201centirely false\u201d but nevertheless \u201calmost started a war\u201d.<\/p>\n<p>The reported sequence of events that unfolded inside the world\u2019s most powerful military is extraordinary to say the least. <\/p>\n<p>Well, not if you\u2019re one of those AI campaigners who have been <a class=\"body-link\" href=\"https:\/\/www.news.com.au\/technology\/innovation\/ai-campaigner-says-ceos-are-freaking-out-about-rapid-advancement-as-humanity-approaches-point-of-no-return\/news-story\/9d9f248496e057d19054a74c2792eaa2?utm_source=chatgpt.com\" target=\"_self\" data-tgev=\"event119\" data-tgev-container=\"bodylink\" data-tgev-order=\"9d9f248496e057d19054a74c2792eaa2\" data-tgev-label=\"technology\" data-tgev-metric=\"ev\" rel=\"nofollow noopener\">warning about this exact scenario for years<\/a>.<\/p>\n<p>MORE: <a class=\"body-link\" href=\"https:\/\/www.realestate.com.au\/news\/data-centres-fuelling-housing-crisis-as-they-win-race-for-land-power-and-water\/?campaignType=external&amp;campaignChannel=syndication&amp;campaignName=ncacont&amp;campaignContent=&amp;campaignSource=newscomau&amp;campaignPlacement=strap\" title=\"www.realestate.com.au\" rel=\"nofollow noopener\" target=\"_blank\">Aus AI boom putting your home at risk<\/a><\/p>\n<p>According to a <a class=\"body-link\" href=\"https:\/\/edition.cnn.com\/\" target=\"_self\" rel=\"nofollow noopener\">CNN<\/a> report citing four internal sources, a special operations analyst fed intelligence about the vessel\u2019s manifest into a chatbot. <\/p>\n<p>The system then combined open-source material with secret signals intelligence and produced the false conclusion that the ship was carrying components connected to a nuclear weapons program.<\/p>\n<p>Want more finance news? <a class=\"body-link\" href=\"https:\/\/onelink.to\/wpmth9\" target=\"_self\" rel=\"nofollow noopener\">Download our new app<\/a> for even more coverage<\/p>\n<p>The analyst then used AI again to package the finding into a neat intelligence report, polishing-up fundamentally unreliable material into authoritative language and then up the chain of command.<\/p>\n<p>\u201cThe internal tools are mostly just copies of the commercial stuff wearing lipstick,\u201d a former senior US official familiar with military and intelligence AI systems said.<\/p>\n<p>The report spread, immediately sparking plans to intercept the ship. Armed personnel were reportedly prepared to board with military aircraft circling overhead before somebody checked the details were actually watertight.<\/p>\n<p>This nauseating little scene is just one of potentially <a class=\"body-link\" href=\"https:\/\/www.news.com.au\/technology\/innovation\/experts-worst-fears-confirmed-as-anthropic-report-reveals-how-ai-is-already-being-used-in-war\/news-story\/9bf097f319bdca1417574c7aa74df3f8\" target=\"_self\" data-tgev=\"event119\" data-tgev-container=\"bodylink\" data-tgev-order=\"9bf097f319bdca1417574c7aa74df3f8\" data-tgev-label=\"technology\" data-tgev-metric=\"ev\" rel=\"nofollow noopener\">billions<\/a> of catastrophic incidents that could arise as humanity leans on artificial intelligence more and more by the day.<\/p>\n<p>The scenario is just one of many problems that inherently emerge from rapid integration, according to the hundreds of computer scientists who have been shouting to the hills about AI since its most primitive iterations.<\/p>\n<p>Only now, after a particularly bizarre case of <a class=\"body-link\" href=\"https:\/\/www.news.com.au\/technology\/innovation\/military\/almost-started-a-war-report-reveals-dark-reality-emerging-as-ai-permeates-civilisation\/news-story\/google.com\/search?q=hugging+face+news.com.au&amp;oq=hugging+face+news.com.au&amp;gs_lcrp=EgZjaHJvbWUyBggAEEUYOTIHCAEQIRigATIHCAIQIRiPAjIHCAMQIRiPAtIBCDQxNTJqMGo3qAIAsAIA&amp;sourceid=chrome&amp;source=chrome.ob&amp;ie=UTF-8\" target=\"_self\" rel=\"nofollow noopener\">autonomous hacking<\/a> followed by frazzled whistleblowers lifting the curtain, has the issue become undeniable.<\/p>\n<p>The latest, potentially devastating, military slip-up has shone light on the tangible risks. <\/p>\n<p>It has provided concrete evidence for those calling immediate regulation, as opposed to yet another vague warning about an all-powerful machine waking up and \u201coptimising us out of existence\u201d.<\/p>\n<p>In this specific case, AI did not need consciousness, or even the control of a bomb. It only really needed to be confidently wrong in a format people were inclined to believe.<\/p>\n<p>Nevertheless, its enormous potential is what keeps the world\u2019s most cashed-up corporations and governments hooked.<\/p>\n<p>Hegseth wants to \u2018unleash experimentation\u2019 in AI war<\/p>\n<p>AI proponents say there\u2019s simply no choice but so soldier ahead and address the risks as they emerge, because the risk of letting adversaries peel away would guarantee a bad outcome. That line of thinking is currently being put under immense strain, despite US President Donald Trump\u2019s insistence that the negative press is a \u201choax\u201d.<\/p>\n<p>In January, US Defence Secretary Pete Hegseth announced an \u201cArtificial Intelligence Acceleration Strategy\u201d intended to transform the Pentagon into an \u201cAI-first\u201d fighting force.<\/p>\n<p>\u201cWe will unleash experimentation, eliminate bureaucratic barriers, focus our investments and demonstrate the execution approach needed to ensure we lead in military AI,\u201d Hegseth said.<\/p>\n<p>The accompanying strategy promised to spread advanced models throughout the military, \u201cdemocratising AI experimentation and transformation across the Department by putting America\u2019s world-leading AI models directly in the hands of our three million civilian and military personnel, at all classification levels\u201d.<\/p>\n<p>That policy deliberately rewards speed, aggressive experimentation and mass adoption. It is the backdrop of the rapidly evolving game of \u201cblink first\u201d playing out between the US and China.<\/p>\n<p>Hegseth declares that \u201cAI is only as good as the data that it receives, and we\u2019re going to make sure that it\u2019s there\u201d. But officials familiar with the system say different internal agencies are using different models under inconsistent standards, with no common process for verifying their output.<\/p>\n<p>Naturally, enemies of the US are also using large language models in attempt to get ahead in the current Middle East spillover. <\/p>\n<p>Just last week, AI giant Anthropic revealed an Iran-linked operator used its popular artificial intelligence model Claude to help track US warships and prepare targeting recommendations.<\/p>\n<p>The \u201cIran-nexus threat actor\u201d compiled targeting handbooks using publicly available information, including ship and aircraft transponder identifiers, commercial satellite-imagery tools and the names of US personnel scraped from military photographs.<\/p>\n<p>The actor also directed Claude to research known vulnerabilities in maritime communications and industrial-control equipment, the company said in its September threat report.<\/p>\n<p>The Pentagon has also clashed with Anthropic over the company\u2019s refusal to remove safeguards intended to stop its Claude models being used for autonomous weapons and mass surveillance. <\/p>\n<p>Anthropic, despite being on the \u201cfrontier\u201d of development, said its systems were simply not reliable enough to be trusted with war.<\/p>\n<p>The reported near-miss exposes the weakness in the promise that human oversight will make it safe. In this case, multiple humans duped by a convincingly authoritative source almost started another unnecessary geopolitical flashpoint.<\/p>\n<p>Accelerating development has been the philosophy that has driven the US economic machine to its current heights. Only now are the broader conceptual risks being given the attention campaigners have been begging for. <\/p>\n<p>Critics, including veteran campaigner and computer scientist Roman Yampolskiy, say the notion of attempting to regulate extremely capable AI is a dangerous misnomer, because it assumes humans will always understand \u2014 and have the collective agency to act on \u2014 the problems that need to be fixed. <\/p>\n<p>At the rate of improvement and the speed at which the current \u201cfrontier\u201d models run, campaigners like Yampolskiy say we are fast approaching a point where it could be impossible to individually parse through each and every mechanical thought process to check it aligns.<\/p>\n<p>\u2018A human rubber stamp\u2019<\/p>\n<p>AI and its use in war has been one of the main issues driving slowdown campaigners.<\/p>\n<p>In 2019, former Pentagon official Paul Scharre warned that \u201covertrusting in machines could lead to accidents and miscalculation, even before a war begins\u201d.<\/p>\n<p>The charmingly-named Stop Killer Robots coalition, formed in 2012, now includes more than 300 organisations calling for meaningful human control over the use of force. <\/p>\n<p>Human Rights Watch warned that same year that formal human oversight could become ineffective as militaries moved towards increasingly automated systems.<\/p>\n<p>Their concerns were often reduced to a cartoon about Terminators roaming battlefields, but the more immediate problem was always much more dull. <\/p>\n<p>In reality, automation bias, compressed decision times and exhausted humans accepting machine recommendations because the \u201ccomputer appeared faster\u201d, are proving just as potentially dangerous.<\/p>\n<p>The Chinese ship incident shows how the idea of \u201chuman control\u201d can become an illusion in itself if the data that informs decision making is entirely made up.<\/p>\n<p> The Red Cross later outlined an almost eerily similar scenario: a human operator \u201claunches an attack based solely on an AI-DSS\u2019s output\u201d, effectively becoming \u201ca human rubber stamp\u201d. <\/p>\n<p>\u2018Approval button does not demonstrate control\u2019<\/p>\n<p>Over the past fortnight, warnings that were once treated as fringe doomsaying have dominated mastheads around the world.<\/p>\n<p>But longtime AI researcher Nate Soares says the current panic has actually come as a relief for safety campaigners, because the things he and his peers have been harping on about for years were finally being discussed at scale.<\/p>\n<p>In his book If Anyone Builds It, Everyone Dies, Soares lays out the tangible risks, but also attempts to explain the ineffable concept of runaway intelligence and the unknowns it will inherently bring.<\/p>\n<p>Following the Hugging Face hacking event, the concept of AI \u201cswarms\u201d breaking free from training environments and rolling through the internet unchecked became realised.<\/p>\n<p>Even the CEOs appeared to be spooked.<\/p>\n<p>Anthropic chief Dario Amodei, OpenAI boss Sam Altman and Elon Musk immediately backed some form of slowdown at the frontier of AI development shortly after.<\/p>\n<p>Their motives and their own role in creating the problem have also <a class=\"body-link\" href=\"https:\/\/www.news.com.au\/finance\/work\/leaders\/big-short-investor-michael-burry-claims-ai-slowdown-warnings-are-concealing-something-else-entirely\/news-story\/d18e60a27112bca69d29898a966c6e22\" target=\"_self\" data-tgev=\"event119\" data-tgev-container=\"bodylink\" data-tgev-order=\"d18e60a27112bca69d29898a966c6e22\" data-tgev-label=\"finance\" data-tgev-metric=\"ev\" rel=\"nofollow noopener\">come under scrutiny<\/a>, namely from Big Short investor Michael Burry.<\/p>\n<p> The argument is that these are companies racing for money, power and market position while asking governments to lock in regulations for the field around them.<\/p>\n<p>For the most worried whistleblowers, however, that is all minutiae. <\/p>\n<p>The frightening world safety campaigners exist in is one where each new problem is treated as an unpredictable development.<\/p>\n<p>Ukrainian AI-governance researcher Anna Mysyshyn was warning US politicians about the reckless implementation of AI in military systems just days before the report on the cargo ship emerged.<\/p>\n<p> \u201cThe approval button alone does not demonstrate the control,\u201d she said.<\/p>\n<p>Another source familiar with US military policy was even more direct.<\/p>\n<p>\u201cAI in targeting is definitely something that is ramping up and there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide,\u201d the source said.<\/p>\n<p>Three Democratic senators \u2014 Mark Warner, Jack Reed and Chris Coons \u2014 have now demanded an inspector-general investigation into whether the Pentagon and intelligence agencies placed AI \u201cexperimentation\u201d ahead of effective governance.<\/p>\n<p>They also want investigators given unrestricted access to other suspected failures in AI-assisted targeting workflows. <\/p>\n<p>Meanwhile, Washington has proposed an AI-incident notification system with Beijing ahead of talks between Donald Trump and Xi Jinping. <\/p>\n<p>Treasury Secretary Scott Bessent said the two powers needed more transparency around failures carrying national-security consequences. <\/p>\n<p>They might just be right.<\/p>\n","protected":false},"excerpt":{"rendered":"Yep, we\u2019re here already. Over the weekend, the US military reportedly rushed aircraft into the air and held&hellip;\n","protected":false},"author":2,"featured_media":900528,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[469105,469119,469155,469089,469099,469137,469151,420599,469129,469110,469097,469120,469086,469128,469113,382282,469157,469108,469150,469145,469102,381272,469116,406913,469092,469139,256,469088,469131,1156,469081,2095,469077,469122,469125,469135,469140,254,255,39410,64,393401,63,469130,452976,469085,14828,469156,469095,469094,399542,228402,469152,469159,197294,268158,343558,469136,469158,469144,469115,469111,469138,3172,466331,254499,469101,469109,90843,93140,469076,469107,469146,27620,469126,469078,469142,175563,391190,19941,10320,20764,37471,469091,395495,469114,14512,28581,364879,469100,469103,402656,469106,27865,469112,469118,455202,469149,1917,469117,469075,469132,8207,469153,469084,469090,136724,41632,208342,105,469143,379588,387659,353806,469121,441165,378293,469098,389955,391600,469104,469127,469093,382511,339485,378923,469123,469161,378343,388274,455197,469147,469082,261826,469079,469133,388083,380113,378575,401301,103578,469154,204813,469134,407848,434892,22514,54798,469083,22637,469141,469080,469148,469160,469124,47360,469087,469096,383052],"class_list":["post-900527","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-that-policy","tag-the-reported-sequence","tag-s-insistence","tag-s-manifest","tag-s-most-cashed-up-corporations-and-governments","tag-s-world-leading-ai-models","tag-a-bad-outcome","tag-a-bomb","tag-a-chatbot","tag-a-chinese-ship-a-chatbot","tag-a-closer-look","tag-a-cnn-report","tag-a-flame","tag-a-format-people","tag-a-former-senior-us-official","tag-a-hoax","tag-a-neat-intelligence-report","tag-a-nuclear-weapons-program-the-operation","tag-a-nuclear-weapons-program-want-more-finance-news","tag-a-particularly-bizarre-case","tag-a-special-operations-analyst","tag-a-war","tag-accompanying-strategy","tag-advanced-models","tag-adversaries","tag-aggressive-experimentation","tag-ai","tag-ai-experimentation-and-transformation","tag-ai-warai-proponents","tag-aircraft","tag-all-classification-levels","tag-america","tag-an-ai-first-fighting-force","tag-an-artificial-intelligence-acceleration-strategy","tag-an-all-powerful-machine","tag-armed-personnel","tag-armed-troops","tag-artificial-intelligence","tag-artificialintelligence","tag-attempt","tag-au","tag-aus-ai-boom","tag-australia","tag-authoritative-language","tag-autonomous-hacking","tag-blink-first","tag-board","tag-bureaucratic-barriers","tag-catastrophic-incidents","tag-civilisation","tag-command","tag-components","tag-computer-scientists","tag-concrete-evidence","tag-consciousness","tag-copies","tag-dark-reality","tag-devastating-ai-prediction","tag-different-internal-agencies","tag-different-models","tag-enemies","tag-even-more-coveragethe-analyst","tag-even-the-control","tag-events","tag-existence","tag-experimentation","tag-four-internal-sources","tag-frazzled-whistleblowers","tag-hegseth","tag-humanity","tag-immediate-regulation","tag-immense-strain","tag-inconsistent-standards","tag-intelligence","tag-its-cargo","tag-its-enormous-potential","tag-its-most-primitive-iterations-only","tag-january","tag-just-one","tag-large-language-models","tag-latest","tag-light","tag-lipstick","tag-little-scene","tag-many-problems","tag-mass-adoption","tag-military","tag-military-ai","tag-military-aircraft","tag-military-and-intelligence-ai-systems","tag-military-slip-up","tag-no-choice","tag-no-common-process","tag-officials","tag-open-source-material","tag-our-investments","tag-our-new-app","tag-our-three-million-civilian-and-military-personnel","tag-plans","tag-polishing-up-fundamentally-unreliable-material","tag-potentially-billions","tag-rapid-integration","tag-report","tag-report-spread","tag-riskaccording","tag-secret-signals-intelligence","tag-soldier","tag-speed","tag-standby","tag-technology","tag-that-line","tag-the-air","tag-the-backdrop","tag-the-chain","tag-the-commercial-stuff","tag-the-curtain","tag-the-data","tag-the-day-the-scenario","tag-the-department","tag-the-details","tag-the-execution-approach","tag-the-false-conclusion","tag-the-finding","tag-the-hands","tag-the-hills","tag-the-hundreds","tag-the-intelligence-report","tag-the-internal-tools","tag-the-issue","tag-the-least","tag-the-military","tag-the-most-powerful-decision-makers","tag-the-negative-press","tag-the-pentagon","tag-the-planet-one-source","tag-the-rapidly-evolving-game","tag-the-report","tag-the-risk","tag-the-risks","tag-the-ship","tag-the-system","tag-the-tangible-risks","tag-the-us","tag-the-us-and-china-hegseth","tag-the-us-military","tag-the-vessel","tag-the-weekend","tag-the-world","tag-their-output-naturally","tag-thinking","tag-this-coming-years","tag-this-exact-scenario","tag-this-specific-case","tag-those-ai-campaigners","tag-us-defence-secretary-pete-hegseth","tag-us-president-donald-trump","tag-years-more","tag-yet-another-vague-warning","tag-your-home"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/900527","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=900527"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/900527\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/900528"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=900527"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=900527"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=900527"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}