{"id":656828,"date":"2026-06-25T03:28:13","date_gmt":"2026-06-25T03:28:13","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/656828\/"},"modified":"2026-06-25T03:28:13","modified_gmt":"2026-06-25T03:28:13","slug":"we-need-an-international-treaty-to-ban-superintelligence","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/656828\/","title":{"rendered":"We Need an International Treaty to Ban Superintelligence"},"content":{"rendered":"

Want to join a Zoom call with Francis Fukuyama to discuss \u201cThe End of History\u201d? Of course you do! Frank is joining our Intellectual Bootcamp<\/a> this week, Friday June 26 at 12 noon. You must be a paying subscriber to participate in the bootcamp\u2014so click here<\/a> to take out a paid subscription today! Once you have subscribed, please email events@persuasion.community<\/a> to be added to the mailing list and receive the Zoom link.<\/p>\n

\"\"<\/a>Anthropic CEO Anthropic Dario Amodei at the G7 Summit on June 17, 2026. (Photo by Anna Moneymaker\/Getty Images.)<\/p>\n

As today\u2019s AI companies race to create artificial intelligence that is smarter than humans, governments around the world are failing to meet the moment.<\/p>\n

Just this April, Anthropic withheld<\/a> its Mythos model from wide release due to its unprecedented cyberattack capabilities. General Joshua Rudd, head of the National Security Agency and Cyber Command, confirmed<\/a> that Mythos \u201cbroke into almost all of our classified systems, not in weeks, but in hours.\u201d This led the Trump Administration to issue an executive order setting up voluntary pre-deployment reviews of AIs. When Anthropic deployed a scaled-back version of Mythos called Fable 5 to the public, the Trump administration used export controls to bar foreign nationals from using the models, forcing Anthropic to disable both Fable 5 and Mythos.<\/p>\n

It\u2019s clear that the U.S. government is starting to take the national security implications of powerful AI seriously. But it is still missing the bigger picture. Pre-deployment evaluations and export controls fall far short of addressing the threats from increasingly capable AIs, because the threats we\u2019re facing aren\u2019t just shockingly capable cyberweapons. It\u2019s much worse than that. The leading AI companies are explicitly trying to build superintelligent AI, or \u201csuperintelligence.\u201d Such AIs would be vastly more capable than humans, fully autonomous, and able to overpower countries\u2019 national security institutions.<\/p>\n

The entire international security architecture is built around the assumption that the most dangerous weapons are controlled by humans, and that the decision to take lives ultimately rests with humans. Superintelligence would upend this. Unlike nuclear weapons\u2014the most lethal weapons we have today\u2014superintelligent systems would not be tools humans can leverage, but rather agents able to pursue their own objectives with no human control over their actions. The leading AI companies are actively worsening the problem by making AIs as autonomous as possible and cutting humans out<\/a> of the process of AI development.<\/p>\n

Superintelligence thus represents the ultimate principal-agent problem<\/a>, and it\u2019s nowhere near being solved. Most alarming of all, Nobel Prize winners, leading AI scientists, and even the CEOs of the top AI companies warn that superintelligence poses an <\/a>extinction risk to humanity. OpenAI\u2019s CEO Sam Altman has repeatedly<\/a> said<\/a> that superhuman machine intelligence is the greatest threat to humanity\u2019s continued existence, yet he affirms<\/a> that OpenAI is \u201cbefore anything else \u2026 a superintelligence research company.\u201d<\/p>\n

As a handful of companies accelerate toward superintelligence, is it already too late to step back from the brink? No. But the window to act is narrow, and it is closing.<\/p>\n

The only solution is an international prohibition<\/a> on the development of superintelligent AI. Governments should pursue this prohibition urgently, because there is no technical solution in sight that would allow humanity to remain in control over AIs vastly smarter and more capable than us.<\/p>\n

\"AIAI Alignment Is Impossible<\/a><\/p>\n

Some argue we can solve this through \u201calignment\u201d research that would let us create AIs that do what humanity wants. But the astonishing growth in AI capabilities is drastically outpacing research into understanding and controlling AIs. To see why that gap matters, consider how the systems are built.<\/p>\n

Right now, engineers essentially grow AI systems out of enormous piles of data, instead of coding the systems line by line like traditional software. They use intense amounts of computational power in large data centers to train these systems. What emerges on the other end of training are AI systems whose internal workings are opaque even to their developers.<\/p>\n

While engineers can test what AI is good at and try to slap some guardrails around it, they do not know how the systems actually work. Dario Amodei, the CEO of AI company Anthropic, has said<\/a> we understand maybe three percent of what is happening inside AI models.<\/p>\n

The inherent unpredictability and uncontrollability of these systems presents massive risks as we get closer to superintelligent systems. And even if one brilliant scientist found a solution, without any government intervention there is no one to stop a different company, or country, from rushing ahead with superintelligent AI that can kill everyone.<\/p>\n

\"DavidDavid Bau on How\u2014and Whether\u2014Artificial Intelligence Thinks<\/a><\/p>\n

Fortunately, it would be possible to both monitor and enforce an international agreement prohibiting superintelligence, because there is a large industrial process behind advanced AI systems. This is akin to the extensive, largely traceable process required to create nuclear weapons. The supply chain is unusually narrow and controlled by Western countries and like-minded democracies. The world\u2019s most advanced AI chips are fabricated almost exclusively by TSMC in Taiwan. The lithography machines required to make those chips are built by a single company, ASML, in the Netherlands. And cutting-edge chip designs come overwhelmingly from one American firm, NVIDIA.<\/p>\n

Additionally, the data centers training and continuously running powerful AIs use power on the scale of small cities. Their footprints are easily tracked using satellite imagery. The Federation of American Scientists recently demonstrated<\/a> this by mapping the buildout of a large-scale AI data center in the United Arab Emirates from 2024 to 2025, using freely available imagery. The size of the industrial process is precisely what makes monitoring, and potential intervention, possible.<\/p>\n

But the window to act is closing. In the coming decade, we can expect algorithms for AI to improve, while hardware like advanced chips could get smaller and more efficient. It\u2019s possible that within a few years, the resources required to build superintelligence could become small enough that monitoring is rendered impossible. This could also eventually make smaller actors, including terrorists and other non-state groups, capable of building such systems.<\/p>\n

We need to act before that happens.<\/p>\n

The United States should domestically prohibit superintelligence and set a policy to prevent its development globally. It should work with other countries to establish an international agreement prohibiting the development of superintelligence using a \u201ctrust but verify\u201d regime, similar to how we have handled nuclear deterrence. While such an agreement is a tall order amid today\u2019s tumultuous geopolitics, it is well within the reach of existing tools of statecraft.<\/p>\n

Countries party to the agreement would ensure that superintelligence is not domestically developed. If anyone outside the group tries to develop it, countries in the group could respond to deter that development. They could use export controls, sanctions, or intelligence cooperation. No country has an interest in any actor, including itself, developing a technology that could overpower the security forces of any government and threaten humanity with extinction.<\/p>\n

None of this will be easy. The ongoing great power competition with China is tense, and there is minimal appetite in Washington for international cooperation. But the same was said about nuclear nonproliferation, and there has been no nuclear detonation in conflict since 1945.<\/p>\n

The political coalition for prohibition is already broader than many realize. In the fall of 2025, leading AI scientists and political figures across the spectrum, like former U.S. National Security Advisor Susan Rice, President Trump\u2019s former adviser Steve Bannon, and former U.S. Chairman of the Joint Chiefs of Staff Admiral Mike Mullen, called<\/a> for a prohibition on the development of superintelligence. Meanwhile, a coalition of more than 150 lawmakers across the UK<\/a> and Canada<\/a> are warning about the extinction-level threat from superintelligence and calling for binding limits, up to a ban<\/a> on developing such technology.<\/p>\n

Yet most policymakers still aren\u2019t taking the danger seriously. They\u2019ve only heard the nice stories of sky-high profitability sold to them by AI executives. What\u2019s missing is both the widespread recognition that superintelligence is an unprecedented national security threat, and the political will to act. The right solution is a doctrine of superintelligence prevention that protects the United States and the rest of the world.<\/p>\n

With it, we can still avert the catastrophe we\u2019re racing toward.<\/p>\n

Andrea Miotti is founder and CEO of ControlAI, a non-profit organization working to keep humanity in control of advanced AI.<\/p>\n

Follow Persuasion on X<\/a>, Instagram<\/a>, LinkedIn<\/a>, and YouTube<\/a> to keep up with our latest articles, podcasts, and events, as well as updates from excellent writers across our network.<\/p>\n

And, to receive pieces like this in your inbox and support our work, subscribe below:<\/p>\n