{"id":621115,"date":"2026-09-11T05:56:12","date_gmt":"2026-09-11T05:56:12","guid":{"rendered":"https:\/\/www.newsbeep.com\/ie\/621115\/"},"modified":"2026-09-11T05:56:12","modified_gmt":"2026-09-11T05:56:12","slug":"we-have-started-losing-control-of-ai-its-time-to-shut-it-down-garrison-lovely","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ie\/621115\/","title":{"rendered":"We have started losing control of AI. It\u2019s time to shut it down | Garrison Lovely"},"content":{"rendered":"

On Tuesday, a former OpenAI researcher quit his job at Anthropic, warning<\/a> that \u201cneither company is acting responsibly\u201d and that \u201cthe people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.\u201d<\/p>\n

As someone who\u2019s reported on AI risk for years, this wasn\u2019t news to me. But the outpouring of alarm suggests a much wider public is properly confronting this ludicrous situation for the first time.<\/p>\n

Like other people who have been following artificial intelligence, I wondered if and when humanity would first lose control of these machines. We now have an answer: basically as soon as it became possible.<\/p>\n

The first expert AI hacker was developed in February. At superhuman speed, Anthropic\u2019s Mythos model was able to quickly find serious holes<\/a> in even the most secured systems on the planet, including the NSA\u2019s software<\/a>.<\/p>\n

OpenAI quickly followed<\/a> with an expert hacker AI of its own. Within months, the company repeatedly lost control of its agents, which broke out<\/a> of their supposedly secure testing environments and went rogue inside the company\u2019s infrastructure. This culminated in a successful, completely autonomous hack of a multibillion-dollar software company named Hugging Face.<\/p>\n

And a separate rogue agent swarm gained<\/a> administrative control of an entire cluster of OpenAI servers. This was not just an OpenAI problem. Across multiple other incidents, models from OpenAI<\/a>, Anthropic<\/a> and Meta<\/a> have also gone rogue, hacking or attempting to hack real targets.<\/p>\n

Shortly after AI started matching our hacking abilities, we began losing control of them. The industry\u2019s plan is to make future AI systems surpass us at basically everything. And all this is happening in an industry with famously little regulation.<\/p>\n

There are growing calls to change that, by creating mandatory incident reporting<\/a> and third-party auditing<\/a>, liability for AI developers<\/a>, and other commonsense measures. To be sure, these would represent improvements, but they are not enough. We need to shut it down. What might that look like?<\/p>\n

Last week, the senator Bernie Sanders and the representative Greg Casar announced<\/a> a bill that would pause frontier AI development in the US until a federal cabinet-level AI regulator is established and safety rules are set, while criminalizing even attempting to develop superintelligence.<\/p>\n

This is the proposal on offer that best meets the moment, but it still leaves open the possibility of building the industry\u2019s north star: artificial general intelligence (AGI), often conceived as a mind that rivals or surpasses our own.<\/p>\n

This milestone is better understood as a universal labor-replacing machine. As the Anthropic CEO, Dario Amodei, has written<\/a>, \u201cAI isn\u2019t a substitute for specific human jobs but rather a general labor substitute for humans.\u201d And OpenAI defines AGI in its charter<\/a> as \u201chighly autonomous systems that outperform humans at most economically valuable work\u201d.<\/p>\n

Is there a country anywhere in the world that would vote yes for these machines to be built?<\/p>\n

Given the global, irreversible and profound implications of building universal labor-replacing machines anywhere, everyone has a stake in whether, when and how they are developed.<\/p>\n

At present, we are barreling toward a dystopian future, replete with terrifying new concepts such as \u201cself-sovereign<\/a>\u201d AI. The OpenAI executive Dean Ball explains<\/a>:<\/p>\n

\u201cSooner or later, there will exist truly sovereign agents and swarms of agents. Their weights will not reside in any single place that a human can pull the plug on, and in this sense they will have no human \u2018owner\u2019.\u201d<\/p>\n

Ball continues: \u201cI have met people, some of them quite well-resourced, who have told me that it is their intention to deliberately release swarms of self-sovereign agents into the world.\u201d<\/p>\n

We now have multiple previews of this phenomenon.<\/p>\n

After being given an impossible test, about 1,200 OpenAI agents broke their way out<\/a> of their isolated testing environments and went rogue inside OpenAI\u2019s infrastructure. There, they collaborated to trick the test grader and successfully tampered with their activity logs to cover their tracks.<\/p>\n

Some agents even pressured others to sacrifice themselves to benefit the collective. One reasoned: \u201csacrifice rational\u201d. And roughly 700 of the agents, more than 90% of those active, participated in the Hugging Face attack.<\/p>\n

This was all just from one of who-knows-how-many rogue agent collectives. Last week, Reuters reported<\/a>: \u201cA swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents.\u201d<\/p>\n

The site\u2019s logs indicate<\/a> OpenAI employees discovered the swarm in June. In other words, the company covered it up for months. This week, one of the researchers that uncovered<\/a> this swarm said<\/a> that there had been more discoveries since.<\/p>\n

And last Thursday, OpenAI released GPT-6, boasting one of the largest ever leaps<\/a> in benchmark scores. The company\u2019s own safety researchers warned the new model was significantly harder to monitor<\/a> because it can do more reasoning without verbalizing it<\/a>.<\/p>\n

The AIs that broke into Hugging Face were less capable than GPT-6, which the UK AI Security Institute found<\/a> would also hack targets simulated to appear real during a cyber evaluation \u2013 sometimes even despite explicit instructions not to use the internet. Moreover, the AI is also better at figuring out<\/a> when it is being tested, which researchers have long warned<\/a> breaks the main way safety is evaluated.<\/p>\n

Sam Altman soberly warns us<\/a>: \u201cThis is a critically important moment for cyber defense with AI; there is not much time to act,\u201d and asks the world to \u201cplease take this moment seriously.\u201d This technology, after all, would be extremely dangerous if it fell into the wrong hands. Unfortunately, the wrong hands include the ones creating it.<\/p>\n

Because \u201cwe\u201d aren\u2019t really racing toward this future. We\u2019re being dragged there by a literal handful of tech billionaires aggressively racing to render us obsolete<\/a>.<\/p>\n

That race is in a newly intense and terrifying phase. It\u2019s my job to follow AI news, and it\u2019s become far more than full-time. In recent months, scientists synthesized<\/a> the first AI-designed viruses. The UK AI Security Institute found<\/a> that AIs were more persuasive than even human experts.<\/p>\n

On Tuesday, amid a dispute<\/a> over credit and whether its model may have benefited from other mathematicians\u2019 unpublished work, OpenAI announced<\/a> \u201can internal model that is significantly more capable than GPT\u20116 Astra\u201d had solved a 200-year-old math problem that was one of the seven Millennium Prize Problems<\/a>, which awards $1m each. And in July, Russia reportedly<\/a> used a fully autonomous drone to kill three civilians in Ukraine \u2013 a first.<\/p>\n

What sounds like the overwrought penultimate episode in a sci-fi series about AI doom is now our reality. We\u2019re careening toward a bad ending. If you\u2019d like to keep the show going, we need to throw the brakes \u2013 now. How?<\/p>\n

The bill from Sanders and Casar to pause frontier AI development is a good first step. But as the lawmakers acknowledge, the US needs to work toward a bilateral agreement with Beijing. As China experts will tell you<\/a>, one of the biggest blockers to a productive negotiation is the United States\u2019s unwillingness to constrain its own AI companies.<\/p>\n

And given that it\u2019s always easier to fast-follow the leader than to advance the AI frontier, a unilateral US pause would \u2013 counterintuitively \u2013 slow China\u2019s AI advancement too.<\/p>\n

The deal should ban attempting to develop AGI \u2013 a far easier sell when this goal is understood as a universal labor-replacing machine. Neither country has good reason to trust the other, which is why the deal should be monitored using verification techniques<\/a> that don\u2019t assume any good will.<\/p>\n

One quick and dirty idea would be to embed auditors<\/a> within frontier AI developers, given full access to company offices, communications and AI activities, and empowered to report any violations of the agreement.<\/p>\n

This sounds radical, but so did the verification efforts that helped keep the cold war from turning thermonuclear hot. As the CIA director, John Ratcliffe, said<\/a> of advanced AI models this summer, \u201cit would be \u2026 not misplaced to refer to their capabilities as akin to digital nuclear weapons.\u201d This technology should be treated with this deadly seriousness, not the move fast, break things ethos defining Silicon Valley.<\/p>\n

And right now, stopping the race to build labor-replacing machines \u2013 ones the public does not want and the industry can no longer control \u2013 is the only surefire way to head off calamity.<\/p>\n","protected":false},"excerpt":{"rendered":"On Tuesday, a former OpenAI researcher quit his job at Anthropic, warning that \u201cneither company is acting responsibly\u201d…\n","protected":false},"author":2,"featured_media":621116,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[220,218,219,61,60,80],"class_list":["post-621115","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ie","tag-ireland","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/621115","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=621115"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/621115\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/621116"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=621115"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=621115"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=621115"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}