{"id":618540,"date":"2026-10-02T14:56:09","date_gmt":"2026-10-02T14:56:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/nz\/618540\/"},"modified":"2026-10-02T14:56:09","modified_gmt":"2026-10-02T14:56:09","slug":"ai-weapons-systems-are-already-here-algorithms-must-not-decide-who-lives-and-dies-kenneth-roth","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/nz\/618540\/","title":{"rendered":"AI weapons systems are already here. Algorithms must not decide who lives and dies | Kenneth Roth"},"content":{"rendered":"

As tech leaders warn<\/a> about the threats of artificial intelligence, most conjure up images of AI agents running amok<\/a>, such as hacking systems used to run our critical infrastructure, banks or even militaries. Yet one serious danger is mentioned less frequently \u2013 the threat posed<\/a> by AI\u2013empowered lethal weapons. Fully autonomous versions of these weapons are colloquially known as killer robots<\/a>.<\/p>\n

Israel\u2019s conduct in Gaza shows that, in an important respect, such AI weapon systems are already here. An algorithm has been deciding who lives and who dies<\/a>.<\/p>\n

For more than a decade, governments have been meeting in Geneva to stop the development of autonomous AI weapons. In early September, 128<\/a> governments met to approve a report<\/a> that could serve as the foundation for a binding treaty.<\/p>\n

But a collection of lawyers sent by Donald Trump and Vladimir Putin teamed up to water it down<\/a>. Notably, they weakened a provision that requires humans to review military targets developed by AI before a strike.<\/p>\n

Fully autonomous weapons remain the focus; the use of AI to determine targets for conventional weapons is not. At risk is the longstanding quest to maintain a \u201chuman in the loop<\/a>\u201d \u2013 that is, meaningful or effective \u201chuman control<\/a>\u201d of AI-empowered weaponry.<\/p>\n

For many people, talk of killer robots conjures up images of Arnold Schwarzenegger\u2019s The Terminator<\/a>. I have often thought about the problem in the context of the Tahrir Square uprising at the height of the Arab spring that led to the ouster of the Egyptian president Hosni Mubarak.<\/p>\n

Mubarak\u2019s fate was sealed<\/a> when the Egyptian army made clear it would refuse<\/a> to fire on Egyptian protesters. But killer robots would have had no such qualms. If Mubarak had been able to deploy them, he might have stayed in power.<\/p>\n

The negotiators in Geneva are not even addressing this problem. The Trump-Putin lawyers made sure that the agreed report referred<\/a> only to war, not repression.<\/p>\n

Today, amid the drone warfare in eastern Ukraine, Russian<\/a> and Ukrainian<\/a> troops are experimenting with fully autonomous weapons to overcome jamming of the signals used to control drones. We can expect these nascent deployments to expand.<\/p>\n

Yet the \u201ckiller robot\u201d metaphor is deceptive. If we look at targeting decisions more expansively, AI has already taken over. In Gaza, the Israeli military is using AI to target Palestinians.<\/p>\n

Israel has long monitored<\/a> phone communications in that territory. After the Hamas attack of 7 October 2023, Israel used an AI system called \u201cLavender<\/a>\u201d to uncover patterns that it equated with the phone of a Hamas operative. People whose phones AI determined met these criteria could be killed.<\/p>\n

Because it was easier to target suspects at home, a separate automated system called \u201cWhere\u2019s Daddy<\/a>?\u201d tracked them there, meaning that their families were often killed too. There was no pretense that Hamas was using its operatives\u2019 homes for military purposes. It was simply easier<\/a> for the AI system to find suspected Hamas fighters where they slept. The families were the collateral damage.<\/p>\n

Theoretically, a human remained in the loop because an AI-picked target was routed through intelligence officers who approved it and sent it to others who launched the attack. In fact, this human role was nominal.<\/p>\n

In the past, Israeli targeting decisions were subject<\/a> to significant review. But in Israel\u2019s killing spree that followed the Hamas attack, the intense demand for targets took precedence over such review.<\/p>\n

Israeli intelligence officers who received the AI selections routinely passed them to the attackers without second guessing. As one officer described<\/a>: \u201cI would invest 20 seconds for each target at this stage, and do dozens of them every day. I had zero added-value as a human, apart from being a stamp of approval. It saved a lot of time.\u201d<\/p>\n

From computer to killing, the human judgment exercised was perfunctory<\/a> \u2013 a \u201crubber stamp<\/a>\u201d. In essence, an algorithm played<\/a> God.<\/p>\n

One Israeli officer suggested<\/a> the practice might be justified because AI makes better decisions than humans do. It is dispassionate, while people\u2019s judgment can be clouded, such as by the animosity toward all Palestinians that some Israelis felt after the Hamas attack.<\/p>\n

As one Israeli soldier explained<\/a>: \u201cI have much more trust in a statistical mechanism than a soldier who lost a friend two days ago. Everyone there, including me, lost people on October 7. The machine did it coldly. And that made it easier.\u201d<\/p>\n

Yet AI algorithms are inevitably affected by human input. How much evidence is required to identify someone as a Hamas fighter? What margin of error is tolerated?<\/p>\n

The answer to such questions depends on the judgment of the AI programmers<\/a> and their commanders \u2013 ordinary Israelis who might have devalued Palestinian civilian life in the fury that followed the Hamas attack. Even the Israeli military conceded<\/a> that about 10% of the human targets slated for assassination were not members of the Hamas military wing at all.<\/p>\n

The Israeli president, Isaac Herzog, epitomized the heedless attitude toward Palestinian civilians when he said<\/a>: \u201cIt is an entire nation out there that is responsible. It\u2019s not true, this rhetoric about civilians were not aware, not involved \u2013 it\u2019s absolutely not true. They could\u2019ve risen up, fought against that evil regime.\u201d<\/p>\n

That same contempt contributed to the Israeli military\u2019s shocking willingness to tolerate civilian deaths. The Israeli military permitted<\/a> an anticipated<\/a> 20<\/a> dead for the killing of a mere suspected low-level Hamas fighter.<\/p>\n

Under the rule of proportionality, international humanitarian law prohibits<\/a> a wartime attack if it \u201cmay be expected to cause incidental loss of civilian life \u2026 which would be excessive in relation to the concrete and direct military advantage anticipated\u201d. Although international law doesn\u2019t specify the number of civilian casualties that are acceptable in a given situation, virtually no legal expert<\/a> considers Israel\u2019s shocking 20-to-one ratio to be lawful.<\/p>\n

These are war crimes<\/a>. They reflect an appalling devaluing of Palestinian civilian life.<\/p>\n

Evoking for me Hannah Arendt\u2019s \u201cbanality of evil<\/a>\u201d, ordinary soldiers became complicit in extraordinary cruelty by trusting what they call the \u201csystem<\/a>\u201d. The essence of that system remained people \u2013 the generals<\/a> who ordered its creation, the military lawyers who excused the unjustifiable, the programmers who tweaked the algorithm, the intelligence officers who passed on the computer-generated targets, and the soldiers who launched the attack.<\/p>\n

But part of what made it easier to trust the system was the pseudo-scientific precision of AI. We are all familiar with the breezy, authoritative tone of today\u2019s chatbots. When AI is used for targeting, it brings a similar feigned omniscience to killing.<\/p>\n

When inevitably flawed human beings make decisions, they can expect others to challenge them. But AI makes it easier for us to defer to its judgments. That deference risks magnifying the human biases that inevitably infect any AI system.<\/p>\n

As we seek to curb killer robots, we should also remain attentive to AI that can lull us into complacency about other targeting. The International Committee of the Red Cross calls<\/a> it \u201cautomation bias\u201d.<\/p>\n

To have meaningful human control of AI, we need not just nominal endorsement of its decisions. We need intense questioning of its product, and accountability for commanders when junior officers fall short.<\/p>\n

Only that skepticism has any hope of ensuring that soldiers who deploy AI remember that real human beings are at the end of the kill chain they enable. Even in war, people should never be snuffed out by the mechanical application of an algorithm.<\/p>\n","protected":false},"excerpt":{"rendered":"As tech leaders warn about the threats of artificial intelligence, most conjure up images of AI agents running…\n","protected":false},"author":2,"featured_media":618541,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,363,364,111,139,69,145],"class_list":["post-618540","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-new-zealand","tag-newzealand","tag-nz","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/618540","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=618540"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/618540\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/618541"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=618540"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=618540"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=618540"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}