That meeting was the first one held after the start of Project Maven, a US Department of Defense initiative using AI to analyze drone surveillance footage. And by late 2017, Maven had a major tech company on board: Google. \u201cThe systems we were talking about were not futuristic,\u201d said Marijan, who is a senior researcher at Project Ploughshares, a peace-focused independent research institute. \u201cThey were existing platforms that had degrees of autonomy in them, or the capability to select and engage targets based on sensor data and sensor input.\u201d<\/p>\n
The world had already seen drone warfare \u2014 deadly machines directed by humans. Now, it was looking at a future where humans may be removed from the loop entirely. \u201cThese were not these Terminator-like figures that we were concerned about, but really what was happening with the enablement of autonomy,\u201d said Marijan.<\/p>\n
The US military has backed AI development for decades, and in turn, AI has transformed warfare<\/p>\n
Nearly a decade later, militaries haven\u2019t yet developed fully autonomous lethal weapons. But these systems sit squarely in the center of a recent high-stakes battle between the US government and AI startup Anthropic. Anthropic is seeking to preserve two \u201cred lines\u201d: bans on domestic mass surveillance and on weapons that can identify, track, and kill targets with zero human involvement. Since the start of the year, it\u2019s emerged as the only military AI contractor to place meaningful limits on what experts call one of the final frontiers of AI warfare.<\/p>\n
But amid shifting alliances, lawsuits, and melodrama, it\u2019s easy to lose sight of the larger context \u2014 that AI is, and long has been, deeply embedded in the military. Seventy years ago, a summer meeting<\/a> between scientists in New Hampshire made the Department of Defense sit up and take notice of AI\u2019s potential for war. Since then, its influence has grown exponentially every decade. In recent years in particular, AI has enabled more and faster killings than ever before.<\/p>\n Even Anthropic seems to think its red lines won\u2019t hold for long. After all, history has proven otherwise.<\/p>\n The US military has backed AI development for decades, and in turn, AI has transformed warfare. In the 2000s, the technology became capable of parsing unprecedented amounts of data collected worldwide, creating a surveillance revolution. And the late 2010s saw the development of advanced facial recognition and other sophisticated machine vision systems.<\/p>\n The fight between Anthropic and the Pentagon has drawn attention to the growing power of these systems. It began in January of 2026, when Defense Secretary Pete Hegseth demanded a renegotiation<\/a> of the DOD\u2019s existing AI contracts. The terms did away with any gray areas or previously agreed-upon terms, allowing the Pentagon to use the companies\u2019 technology within the vague and expansive limits of \u201cany lawful use.\u201d Anthropic \u2014 the only AI company approved to deploy its tech on the Pentagon\u2019s classified networks at that point \u2014 objected.<\/p>\n Even Anthropic seems to think its red lines won\u2019t hold for long<\/p>\n A contractor like Anthropic setting limits on specific uses of its tech is unusual. \u201cIt\u2019s not government-created technology in the way that the Manhattan Project was,\u201d nor a conventional military supplier like Northrop Grumman, said Andrew Reddie, an associate research professor of public policy at the University of California, Berkeley. \u201cThis is one of the pain points that\u2019s made clear when you\u2019ve got this startup ecosystem engaging with the Pentagon directly.\u201d Even within Silicon Valley, he said, there\u2019s \u201ca lot of disagreement\u201d over when limits should be set.<\/p>\n It\u2019s also unclear who will win. As a negotiation tactic, the DOD designated Anthropic a military supply chain risk in March, and President Donald Trump declared he was banning all government agencies from using its Claude system. The relationship has apparently warmed somewhat<\/a> since then, with the release of Anthropic\u2019s cybersecurity-focused model Mythos, but a court battle is still playing out. Anthropic declined to provide a comment for this story.<\/p>\n Either way, the debate has brought \u201cfully autonomous weapons\u201d into the public lexicon in a new way. But AI\u2019s creeping influence in military operations hasn\u2019t slowed down in decades.<\/p>\n \u201cWe\u2019ve kind of crossed the rubicon while we pretend that we haven\u2019t,\u201d Reddie said.<\/p>\n At the center of the debates is DOD Directive 3000.09, one of the only policies governing the use of lethal autonomous weapons. Originally written in 2012<\/a>, it defines such a system as one that, \u201conce activated, can select and engage targets without further intervention by an operator.\u201d And it decrees that both fully autonomous and semi-autonomous weapons be designed to allow humans to \u201cexercise appropriate levels\u201d of judgment over the use of force.<\/p>\n The directive set up the \u201cfirst policy on the use of autonomy in warfare,\u201d said Hamza Chaudhry, who leads AI and national security at the Future of Life Institute.<\/p>\n \u201cEven without full autonomy, AI compresses kill chains to mere seconds.\u201d<\/p>\n Depending on how you interpret the definition, however, certain missile defense programs may have crossed that line decades ago. Take the Phalanx CIWS, for instance. It\u2019s an automated weapon system resembling a very large gun, built to defend naval vessels from incoming missile attacks. That type of system wouldn\u2019t work if there were a human in the loop, since it has to respond in milliseconds.<\/p>\n The difference, some experts say, is that these systems operate solely in a defense-only, fixed environment. They\u2019re engaging, this interpretation goes, but not deciding \u2014 just reacting to an incoming threat. \u201cThe \u2018and\u2019 is doing a lot of work inside of that statute \u2014 we have systems that can decide and systems that can engage but you can\u2019t have a system that does both,\u201d Reddie said.<\/p>\n \u201cEven without full autonomy, AI compresses kill chains to mere seconds so that humans are not actually making the assessments that international humanitarian law requires to prevent civilian harm,\u201d said Maddy Batt, legal fellow at Tech Justice Law. \u201cWhen humans\u2019 failure to do that results in civilian death, that is a war crime.\u201d<\/p>\n The definition between offensive and defensive, too, is fuzzy. \u201cOne of my favorite exercises with my students is putting up a military technology and then asking, \u2018Is this defensive or offensive?\u2019\u201d Reddie said, pointing to the example of a nuclear weapon in a silo, which some would argue is defense because it\u2019s being used to deter, and some would argue is offense because it\u2019s designed to hit foreign targets. \u201cJust because its primary function \u2026 is defensive in nature doesn\u2019t mean the technology itself is defensive in nature.\u201d<\/p>\n Certain missile defense systems may have crossed the line of autonomous response decades ago<\/p>\n Sorin Adam Matei, a professor at Purdue University, was blunter: \u201cYou cannot fight a war only in defense.\u201d<\/p>\n In 2023, the government\u2019s Chief Digital and Artificial Intelligence Office (CDAO) \u2014 which is the central hub of all the DOD\u2019s AI operations \u2014 issued an update<\/a> to DOD Directive 3000.09. But it didn\u2019t resolve the document\u2019s core ambiguities. In 2024, the Biden administration published a memorandum<\/a> on AI and national security, setting up rules for how AI can be used in certain national security scenarios \u2014 and for now, even under Trump, that policy is in force. But the Pentagon has seen significant upheaval. The CDAO is currently undergoing a significant restructuring that makes it more isolated from the rest of the DOD, and the office now reports to Emil Michael, who is both the DOD\u2019s undersecretary of research and engineering and the department\u2019s CTO.<\/p>\n International efforts, such as the Convention on Certain Conventional Weapons and the Campaign to Stop Killer Robots, haven\u2019t been able to make much progress either. Marijan told The Verge that though the CCW has greatly helped smaller countries to understand the landscape around AI warfare, progress overall has \u201cbeen very slow and we haven\u2019t seen concrete agreement, particularly among the major countries and the more sophisticated militaries.\u201d Though some countries have expressed interest in a ban on lethal autonomous weapons, there\u2019s not even an official international definition of the term. The situation \u201coften results in a lot of people talking past each other \u2026 and some countries find the lack of a binding instrument to be to their advantage,\u201d said Sarah Shoker, a senior research scholar at the University of California, Berkeley and former lead of OpenAI\u2019s geopolitics team.<\/p>\n Some countries have expressed interest in a ban on lethal autonomous weapons, but there\u2019s not even an official international definition<\/p>\n \u201cI think most people \u2014 policymakers, civil society members \u2026 who attend these meetings are likely tired,\u201d Shoker added. \u201cIt\u2019s been over a decade, and there is really no agreement.\u201d<\/p>\n Whatever the status of autonomous weapons before 2017, Maven accelerated their timeline dramatically \u2014 and changed the terms of debate. The program marked the first time the idyllic, \u201cdon\u2019t be evil\u201d Silicon Valley of the 2010s was obviously involved in warfare, forcing Google employees and the public to react. \u201cThat was the first national conversation we had about the role of AI technologies in military operations,\u201d said Matei.<\/p>\n Project Maven began with a memo from Robert Work, then deputy defense secretary. He established an \u201cAlgorithmic Warfare Cross-Functional Team\u201d tasked with training AI algorithms that could be deployed to war zones<\/a>, maybe even by year-end. Google was soon awarded the contract.<\/p>\n Internal resistance grew quickly. In April 2018, about 4,000<\/a> employees signed a petition demanding that Google remove itself from the \u201cbusiness of war.\u201d The letter said a Google executive had assured Googlers their tech \u201cwill not \u2018operate or fly drones\u2019 and \u2018will not be used to launch weapons.\u2019\u201d But \u201cwhile this eliminates a narrow set of direct applications, the technology is being built for the military, and once it\u2019s delivered it could easily be used to assist in these tasks,\u201d the employees said. \u201cWe cannot outsource the moral responsibility of our technologies to third parties.\u201d Google did not respond to a request for comment for this story.<\/p>\n Recently, MSS has reportedly been integral<\/a> to both the capture of Venezuelan President Nicol\u00e1s Maduro and the US strikes on Iran, which have reportedly<\/a> killed thousands to date in the region.<\/p>\n \u201cYou cannot fight a war only in defense.\u201d<\/p>\n Google employees argued their company should take a stand \u2014 and it did, choosing not to renew its contract amid the controversy in mid-2018. But Amazon and Microsoft quickly swooped in to pick up tens of millions of dollars in contracts for the same work. Palantir soon took over, and Project Maven became the Maven Smart System (MSS), which not only allows for object detection and tracking but also analyzing surveillance data on a large scale.<\/p>\n Claude is intertwined with MSS too, albeit in a more limited capacity. After Anthropic began allowing military use of its technology in 2024, its Claude chatbot was added to the user interface, helping analysts query the system about certain geographical areas and types of intelligence or targets. Even this relatively minor addition, some experts say, has made the system more efficient. As a result, Claude enabled targeting, potentially striking many more individuals.<\/p>\n The sheer volume of targets could make any meaningful human supervision difficult, said Shoker. \u201cWhat we know about MSS is that it reduces the number of human beings in the targeting cycle \u2014 and that\u2019s actually by design.\u201d<\/p>\n While Anthropic might have been all right reducing human intervention, it\u2019s pushed back against setting it to zero. As Google found with Project Maven, though, competitors are more than willing to fill the gap.<\/p>\n Other AI companies have jumped at the chance to work with the Pentagon. Even before its fight with Anthropic began, several major AI labs had loosened their operating guidelines<\/a> or tweaked their mission statements<\/a> to allow for military deals \u2014 including Google, OpenAI, and arguably Anthropic itself<\/a>. OpenAI quickly signed onto the terms<\/a> Anthropic had spurned. And in the months after snubbing Anthropic, the Department of Defense signed deals with eight companies to deploy their AI on classified networks: Google, Microsoft, Amazon Web Services, Nvidia, OpenAI, Reflection, Oracle, and SpaceX.<\/p>\n AI companies have jumped at the chance to work with the Pentagon<\/p>\n Shoker said that while working at OpenAI \u2014 one of the sector\u2019s biggest players \u2014 she didn\u2019t interact with many people studying the risks of decision-support systems or autonomous weapons systems. The research community in those areas is still \u201cextremely siloed,\u201d she said. \u201cPeople in this space have been warning and researching about [those risks] for a while,\u201d Shoker said, adding, \u201cThe problem \u2026 is that those people were not in the labs.\u201d OpenAI did not respond to a request for comment.<\/p>\n Internally, employees have voiced dissent<\/a> in ways that strongly echo the pushback against Project Maven almost a decade ago. Externally, OpenAI\u2019s Pentagon deal sparked widespread controversy, leading CEO Sam Altman to announce<\/a> he was attempting to renegotiate the deal. But Silicon Valley executives are aggressively pushing back against employee organizing and speaking out, including by using AI to identify leakers<\/a>. And many tech workers already fear for their jobs in an era when AI is set to replace entry-level roles at their own firms.<\/p>\n Meanwhile, through its court fight with the Trump administration, Anthropic is still trying to regain its position as a military and intelligence contractor. Its technology has been deeply enmeshed in the DOD\u2019s operations since 2024<\/a>, and last summer it introduced Claude Gov<\/a>, a product designed specifically for use by national security agencies with loosened guardrails around classified materials.<\/p>\n Anthropic is risking a high-stakes loss as it reportedly prepares to IPO this year. It\u2019s reportedly in talks with investors to raise funding at a $900 billion<\/a> valuation. There\u2019s more investor pressure for the lab to turn a profit than ever before<\/a>.<\/p>\n In recent months, Anthropic CEO Dario Amodei has held firm on mass surveillance for Americans, but he\u2019s demonstrated no problem with \u2014 and in fact expressed his support for<\/a> \u2014 such surveillance for everyone else.<\/p>\n Anthropic\u2019s \u201cvery narrow\u201d red lines \u201cdo not go far enough to protect human rights or to comply with international law,\u201d said Tech Justice Law\u2019s Batt. \u201cAnthropic specifically talks about mass domestic surveillance of US persons as posing grave civil liberties concerns, but the same civil liberties concerns apply with equal force to non-US persons,\u201d she added. \u201cIn fact, we should probably be especially concerned, given the brutal immigration crackdown, about advanced capabilities for mass surveillance of non-citizens.\u201d<\/p>\n Anthropic CEO Dario Amodei has decried mass surveillance for Americans but demonstrated no problem with such surveillance for everyone else<\/p>\n That doesn\u2019t even cover the ethics of mass surveillance abroad, an issue that is strikingly absent from most ethics conversations happening stateside.<\/p>\n And while Amodei believes Anthropic\u2019s systems aren\u2019t ready to support fully autonomous weapons, he\u2019s not conceptually or morally opposed to them. In a blog post<\/a>, he said that \u201cfully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense.\u201d Amodei even said he was happy to \u201cwork directly with the Department of War on R&D to improve the reliability of these systems\u201d and speed up the timeline for the company\u2019s help in deploying them.<\/p>\n That position, Tech Justice Law\u2019s Batt said, is \u201cfundamentally in tension\u201d with international humanitarian law, which requires case-by-case assessments of any given attack. \u201cThe adoption of AI into these targeting processes is really based on a \u2018speed wins\u2019 ethos,\u201d she said. \u201cRegardless of when in the process humans are looped in, if the goal of implementing these systems is to transform something that took weeks or days into seconds, humans are not meaningfully making that assessment for themselves.\u201d<\/p>\n When Hegseth announced his intent to redraft the Pentagon\u2019s AI contracts, he explicitly stated that the DOD \u201cmust accept that the risks of not moving fast enough outweigh the risks of imperfect alignment\u201d \u2014 and that a significant amount of testing and evaluation must be done away with.<\/p>\n \u201cThe adoption of AI into these targeting processes is really based on a \u2018speed wins\u2019 ethos.\u201d<\/p>\n Past administrations have sought to remove \u201cpurely bureaucratic\u201d layers of warfare while preserving safety, Chaudhry said, but the Hegseth memo \u201cexplicitly sets up a tradeoff and says that we favor speed.\u201d This isn\u2019t typically the best blanket approach when it comes to military operations, experts say, but it is often the deadliest. AI has already enabled that lethality \u2014 and it\u2019s poised to go even further.<\/p>\n Since the Hegseth memo, says Marijan, more countries are paying attention to the risks of autonomous warfare. But against the American military and its partners in Silicon Valley, it\u2019s not clear how much they can do. Right now, they\u2019re simply trying to understand its implications. If humans are \u201cjust rubber-stamping decisions,\u201d Marijan said, \u201cwhat are the lines of accountability there \u2014 and what does the liability look like?\u201d<\/p>\n During a Defense One Tech summit in July 2017, where Maven was centered as the future of the department, the chief of the Algorithmic Warfare Cross-Functional Team presented<\/a> to an audience.<\/p>\n AI, he said, would not be selecting a target in combat \u201canytime soon.\u201d But now, without intervention, that moment isn\u2019t far off.<\/p>\n Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.Hayden FieldClose Hayden Field<\/p>\n Posts from this author will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n See All by Hayden Field<\/a><\/p>\n AIClose<\/p>\n AI<\/p>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n See All AI<\/a><\/p>\n AnalysisClose<\/p>\n Analysis<\/p>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n See All Analysis<\/a><\/p>\n AnthropicClose<\/p>\n Anthropic<\/p>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n See All Anthropic<\/a><\/p>\n LawClose<\/p>\n Law<\/p>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n See All Law<\/a><\/p>\n OpenAIClose<\/p>\n OpenAI<\/p>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n See All OpenAI<\/a><\/p>\n PolicyClose<\/p>\n Policy<\/p>\n Posts from this topic will be added to your daily email digest and your homepage feed.<\/p>\n FollowFollow<\/p>\n![\"An](\"https:\/\/www.newsbeep.com\/ca\/wp-content\/uploads\/2026\/05\/268475_AIs_red_line_JMac5.png\"\/)
<\/a><\/p>\n![\"Hayden](\"https:\/\/www.newsbeep.com\/ca\/wp-content\/uploads\/2026\/05\/HAYDEN_BLURPLE.jpg\"\/)
<\/p>\n