{"id":76785,"date":"2025-08-12T08:52:05","date_gmt":"2025-08-12T08:52:05","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/76785\/"},"modified":"2025-08-12T08:52:05","modified_gmt":"2025-08-12T08:52:05","slug":"building-trust-in-military-ai-starts-with-opening-the-black-box-war-on-the-rocks","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/76785\/","title":{"rendered":"Building Trust in Military AI Starts with Opening the Black Box \u2013 War on the Rocks"},"content":{"rendered":"

At high-profile TED-style events with millions of online viewers, charismatic tech CEOs take the stage to unveil their latest gadgets<\/a>. In slick, high-energy presentations, they criticize the slow and outdated cycles of traditional military innovation and procurement. They paint vivid scenarios of future warfare<\/a> where adversaries are agile, tech-savvy, and armed with autonomous systems, implying that only the most cutting-edge technology can prevail. Unsurprisingly, the solution to these looming threats aligns perfectly with the products they happen to be selling. Policymakers, often unfamiliar with the complexities of AI<\/a> and emerging tech, along with senior military officials<\/a> and influential defense industry figures, are drawn in by the urgency and clarity of the message. Back home, they confidently explain to others why preparing for killer drone swarms and AI-powered adversaries justifies immediate investment in these high-tech solutions.<\/p>\n

However, there appears to be a noticeable gap between the capabilities often portrayed by the defense tech industry, through systems such as Palantir\u2019s Gotham<\/a>, Avalor\u2019s Nexus<\/a>, or Helsing\u2019s Altra<\/a>, and what can realistically be achieved in complex military operational environments. In our own academic research on mission planning for military platforms, for example, we frequently encounter complex mathematical challenges<\/a> related to optimizing human-machine interaction in an operationally realistic, practically usable, and responsible way. Addressing these challenges often requires novel methods for making systems smarter and more adaptive.<\/p>\n

One such method is\u00a0Bayesian optimization, a type of mathematical problem-solving used to improve complex systems through trial and error. Imagine tweaking a system based on user feedback, trying something, seeing how it performs, and then using that result to make a better decision the next time. That\u2019s essentially what Bayesian optimization does: It helps a system learn, step by step, to make better choices, even when information is limited or uncertain. This approach often relies on something called a\u00a0Gaussian process, a statistical method that makes educated guesses about things like user preferences or system behavior, based on the data that\u2019s already available.<\/p>\n

However, these models can be difficult to interpret and are often built on certain assumptions. One key assumption is that the relationships or patterns in the data are smooth, meaning that small changes in input variables will lead to small, predictable changes in outcomes. In other words, the system assumes that the world behaves in a gradual and continuous manner. This assumption of smoothness can be problematic in complex and dynamic real-world environments like military operations, where changes may be abrupt, relationships nonlinear, and data noisy or incomplete. To make these techniques work in practice, you need more than just smart algorithms. You need models that can handle uncertainty, adapt quickly, work with incomplete or noisy data, while also remaining \u00a0transparent, explainable and interpretable to human users. Developing such systems requires deep technical innovation, rigorous testing, and a lot of iteration.<\/p>\n

Here lies the dilemma: Companies that have the resources to build these advanced systems often face two choices. Either they\u00a0choose not to adapt these techniques to settings that make them transparent and explainable, or they\u00a0do use and adapt them, but keep their methods proprietary and opaque, often out of fear of losing competitive advantage. While this is understandable from a commercial perspective, it poses a serious challenge for defense organizations, which need a clear understanding of what is happening\u00a0under the hood\u00a0in order to ensure that AI is used safely, ethically, and in alignment with strategic goals.<\/p>\n

While industry plays a vital role and should be part of this solution, its incentives, focused on speed, scalability, and market dominance, don\u2019t always align with the public interest. Governments should often move more cautiously and transparently, prioritizing responsible AI adoption over rapid deployment. For this reason, collaboration between public and private sectors is crucial, but it should be shaped by democratic values, accountability, and the specific needs of public safety and ethics, not just by commercial logic. A potential way forward could involve hybrid arrangements in which proprietary systems remain closed to the general market but are made open and auditable to trusted government partners under strict contractual safeguards. This is not<\/a> a new<\/a> idea<\/a>, but what is new is the growing difficulty<\/a> of understanding and verifying the behavior of modern AI systems due to their black- or gray-box nature. Even in relatively constrained domains like object detection (used for instance, in target recognition), explainability<\/a> remains highly sensitive to the model architecture, training data, and context of deployment, and often degrades under data shift or domain adaptation.<\/p>\n

Similarly, in the domain of drone swarm decision-making under adversarial conditions, where drones must explore environments and dynamically allocate tasks, researchers at the Dutch Ministry of Defense\u2019s Data Science Centre of Excellence face complex, game-theoretical challenges for which no optimal solutions currently exist. We are actively<\/a> addressing above mentioned challenges in collaboration with a team of researchers, the military, and industry. Thus, developing, validating, maintaining, and deploying robust and operationally viable AI systems is significantly more challenging than current marketing narratives seem to suggest. Achieving this requires sustained technical expertise<\/a>, rigorous testing<\/a>, and continuous oversight<\/a>.<\/p>\n

As militaries increasingly turn to industry for AI capabilities, governments should not only guard against the risk of strategic lock-in<\/a>, but more urgently, against inflated expectations and insufficient scrutiny. These factors can lead to both tactical and strategic failure. AI is not a turnkey solution to military dominance. It is a complex, rapidly evolving capability that demands strategic patience, institutional expertise, rigorous oversight, and a deep understanding of its inner workings. While it may be tempting to channel NATO\u2019s expanding defense budget<\/a>s into off-the-shelf AI solutions, overreliance on such approaches could become a strategic liability.<\/p>\n

This is not to suggest that militaries should avoid collaborating with industry. On the contrary, effective military AI development requires sustained engagement across ecosystems<\/a> that include industry and civilian academia, particularly in the light of the current scaling challenges<\/a> and AI\u2019s inherently dual-use nature. However, participation in such ecosystems should not be mistaken for simply procuring off-the-shelf products.<\/p>\n

True collaboration demands that the military cultivates robust internal expertise (AI literacy)<\/a> across all levels of the organization, boosting the human readiness level. This is especially important when evaluating advanced AI platforms, as government officials should be equipped to critically assess their capabilities, limitations, and the conditions under which they can be used responsibly. At the same time, many industry and academic partners \u2014 though not all \u2014 lack deep operational military insight. Their defense literacy should similarly be strengthened. In this sense, advancing military AI is not just about building better systems, it\u2019s about cultivating better ecosystems.<\/p>\n

While industry and academia may drive faster advances in AI technology<\/a>, the military possesses indispensable domain expertise about the realities and requirements of operational environments. This expertise is essential not only for evaluating how AI systems perform under the complex and often ambiguous conditions of military operations, but also for determining whether their use aligns with principles of responsible and lawful deployment.<\/p>\n

Consider, for example, the use of autonomous aerial drone swarms for military intelligence operations. Technologies that enable autonomous collision avoidance<\/a> and autonomous navigation in resource-scarce environments are undoubtedly important but they only represent one part of the puzzle. What is the real-world performance of object detection models used for target identification under conditions of data drift? What level and type of explainability is required for these models and is understandable for the end-user (and\/or commander), given the specific operational and legal context? What constitutes an effective search strategy for a heterogeneous swarm, especially under constraints like limited flight endurance and the potential presence of adversarial countermeasures?<\/p>\n

These are not just technical questions but operational ones that require deep military expertise. That expertise is essential not only for conducting applied research but also for formulating the right research questions in the first place. This process often reveals a disconnect between the actual state of the art in AI and the aspirational narratives marketed by industry. Companies tend to sell ideas, prototypes, and concepts \u2014 not fully integrated, validated, and mission-ready solutions. This is not inherently problematic, but off-the-shelf ideas are no substitute for informed, mission-driven development. Militaries should not mistake these offerings for complete, turnkey capabilities. As of now, such comprehensive solutions do not exist.<\/p>\n

To move toward them, militaries should do more than procure systems: They should build a foundation of expertise that enables meaningful collaboration with external stakeholders. Without that internal capacity, even the best partnerships with industry and academia will fall short.<\/p>\n

So, What Now? <\/p>\n

First, militaries should invest in building a strong internal knowledge base in data science and AI, one that is firmly rooted in operational realities. This effort should not occur in isolation. Instead, it should take place within vibrant ecosystems<\/a> that include both industry and civilian academia. We are not advocating for full autonomy from external partners, but we do caution against overreliance and naive procurement.<\/p>\n

Second, given the current academic state of the art in AI, defense organizations should resist the urge to accelerate deployment<\/a> out of fear of falling behind other states. Fear clouds judgment. Militaries have both the opportunity and the responsibility to proceed with deliberation. Rushing into AI adoption without a clear understanding, oversight, and accountability mechanisms risks undermining both mission effectiveness and public trust.<\/p>\n

Developing military AI that is effective, robust, and responsible requires the active involvement of military experts, and it takes time. Such development demands a foundational level of AI literacy within the military, and defense literacy within industry and academia. Without both AI and defense literacy, responsibility and effectiveness could be presented as opposites<\/a>. On the contrary, responsible use of AI is a prerequisite for effective use. In the long run, irresponsible deployment of AI will prove not only unethical but also operationally unsound.<\/p>\n

Ultimately, both effective and responsible use of AI depend on a clear and informed understanding of how systems are designed, developed, and deployed. The good news: Investing in AI literacy within the military and defense literacy among external stakeholders serves both aims. True dominance in this new domain will not be won by haste or spectacle, but by those who carry the burden of understanding, and the courage to wield it wisely.<\/p>\n

\u00a0<\/p>\n

Roy Lindelauf, PhD, is a full professor in data science in military operations at the War Studies Department of the Netherlands Defence Academy. He also holds an endowed chair in data science for safety and security at the Department of Cognitive Science and Artificial Intelligence at Tilburg University. He leads the Data Science Centre of Excellence of the Dutch Department of Defense. <\/p>\n

Herwin Meerveld is an officer in the Royal Netherlands Air and Space Force. He is coordinator of the Data Science Centre of Excellence of the Dutch Department of Defense. He is an external PhD candidate at Tilburg University. <\/p>\n

Image: Midjourney<\/p>\n","protected":false},"excerpt":{"rendered":"At high-profile TED-style events with millions of online viewers, charismatic tech CEOs take the stage to unveil their…\n","protected":false},"author":2,"featured_media":76786,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":["post-76785","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/76785","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=76785"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/76785\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/76786"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=76785"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=76785"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=76785"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}