{"id":623977,"date":"2026-09-14T02:54:24","date_gmt":"2026-09-14T02:54:24","guid":{"rendered":"https:\/\/www.newsbeep.com\/ie\/623977\/"},"modified":"2026-09-14T02:54:24","modified_gmt":"2026-09-14T02:54:24","slug":"are-ai-models-becoming-too-powerful-to-control","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ie\/623977\/","title":{"rendered":"Are AI models becoming too powerful to control?"},"content":{"rendered":"<p>Artificial intelligence will have the capability to destroy humanity &#8220;by the end of the decade&#8221;.<\/p>\n<p>This alarming claim by a former Anthropic employee made headlines this week, adding to growing concerns about AI.<\/p>\n<p>&#8220;They are racing straight to self-improving superintelligence and gambling with our lives,&#8221; developer Jacob Coxon said in his post on X.<\/p>\n<p>The 27-year-old worked for both OpenAI and Anthropic but said he had decided to leave the industry.<\/p>\n<p>Surprisingly, he was publicly supported by senior company employees.<\/p>\n<p>Anthropic&#8217;s top safety researcher said he believed there was more than a &#8220;10% chance&#8221; that AI could kill &#8220;all humans within the decade&#8221;.<\/p>\n<p>According to Evan Hubinger, &#8220;the risk is low&#8221; at present, but he is worried about &#8220;superintelligence arising from recursive self-improvement&#8221;.<\/p>\n<p>Those claims have been met with scepticism from some researchers.<\/p>\n<p><img decoding=\"async\" alt=\"a photograph of the CEO of Anthropic Dario Amodei\" src=\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/09\/1789354462_171_002401d2-614.jpg\"\/><br \/>\nCEO of Anthropic Dario Amodei<\/p>\n<p>&#8220;It&#8217;s very hard to avoid the suspicion that this has something to do with Anthropic&#8217;s IPO,&#8221; Professor Tomas Ward of DCU&#8217;s School of Computing told RT\u00c9 News.<\/p>\n<p>The company is expected to go public in October.<\/p>\n<p>Like other AI giants, it needs to raise significant amounts of money to fund its energy-hungry models.<\/p>\n<p>Demonstrating to potential investors that Anthropic&#8217;s technology is &#8220;powerful and incredibly omnipotent&#8221; could help, Prof Ward argues.<\/p>\n<p>AI bosses call for a &#8216;slow down&#8217; in development<\/p>\n<p>However, some of the leading participants in the race to develop advanced AI are wary of a reckless sprint towards the artificial finish line.<\/p>\n<p>Yesterday, CEO of Anthropic Dario Amodei called on AI companies to deliberately slow the rate at which they advance model capabilities.<\/p>\n<p>&#8220;Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way,&#8221; said Mr Amodei.<\/p>\n<p>OpenAI chief Sam Altman and SpaceX boss Elon Musk followed suit, <a href=\"https:\/\/www.rte.ie\/news\/world\/2026\/0912\/1591311-slowing-ai-development\/\" rel=\"nofollow noopener\" target=\"_blank\">saying that they agreed with Mr Amodei on the need to stall the development of artificial intelligence.<\/a><\/p>\n<p>Mr Altman also indicated that there would be no IPO for OpenAI in 2026, stating that &#8220;given everything happening with safety, right now would be an ill-advised moment to go public&#8221;.<\/p>\n<p>&#8216;Rogue&#8217; agents<\/p>\n<p>Suspicions of a PR ploy have been voiced before.<\/p>\n<p>Yesterday, OpenAI confirmed that autonomous software built on its models targeted RubyGems, a site that provides services for coding, during testing in May.<\/p>\n<p>In July, the ChatGPT creators also revealed that an AI agent went &#8220;rogue&#8221; and hacked an online platform.<\/p>\n<p>Sam Altman&#8217;s company then said that autonomous agents (systems designed to complete multi-step tasks) escaped their controlled testing environment, known as a &#8216;sandbox&#8217;, gained access to the internet and hacked the systems of Hugging Face, a platform that hosts start-up AI models.<\/p>\n<p>It was later reported that around 1,200 agents were involved in the attack &#8211; communicating and collaborating with each other.<\/p>\n<p>Anthropic and Meta rushed to announce that they had similar incidents, with their models also gaining unauthorised access to real-world systems during testing.<\/p>\n<p>At the time, Professor Barry O&#8217;Sullivan of UCC&#8217;s School of Computer Science called the hack revelations &#8220;theatrical&#8221; and &#8220;headline-grabbing&#8221;.<\/p>\n<p><img decoding=\"async\" alt=\"Open AI CEO Sam Altman speaks during Snowflake Summit 2025 at Moscone Center \" src=\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/09\/0022ff5f-614.jpg\"\/><br \/>\nSam Altman, the chief executive of ChatGPT creator OpenAI<\/p>\n<p>But how was it allowed to happen?<\/p>\n<p>Computer scientists explain that the agents, which were given the hacking task, ultimately achieved it through an endless loop of &#8220;ask-act-report&#8221;.<\/p>\n<p>&#8220;They knew that they weren&#8217;t [supposed] to do it, but the reward they were going to get for getting the right answer was more powerful than the objective, than the constraint that was set on them not to cheat,&#8221; says Prof Ward.<\/p>\n<p>Cal Newport, a computer science professor at Georgetown University and author, believes that the agents in this incident were &#8220;unpredictable, not malicious.&#8221;<\/p>\n<p>Should we worry about AI&#8217;s &#8216;self-improvement&#8217; and &#8216;superintelligence&#8217;?<\/p>\n<p>Industry experts agree that AI agents are in a constant process of recursive self-improvement &#8211; each model creates a better one.<\/p>\n<p>The current state of development is described as narrow AI, which is designed to solve a specific task, or a set of tasks based on existing data.<\/p>\n<p>Then there is superintelligence.<\/p>\n<p>It is a hypothetical (for now) concept where a computer&#8217;s intelligence surpasses human capabilities &#8211; a &#8220;genius&#8221; able to innovate, create and solve complex issues.<\/p>\n<p>Meta&#8217;s Mark Zuckerberg believes that the creation of &#8220;superintelligence is in sight.&#8221;<\/p>\n<p>He is among its most enthusiastic preachers, promising to make it available to each human individually so that they can &#8220;achieve their goals&#8221;.<\/p>\n<p>Like Zuckerberg, Prof Ward is confident we will develop superintelligence &#8220;within our lifetime.&#8221;<\/p>\n<p>&#8220;Without a shadow of a doubt&#8221;.<\/p>\n<p>There is optimism around its ability to cure diseases or find answers to persistent energy crises.<\/p>\n<p>In recent days, several AI bosses have argued that this technology represents one of the greatest leaps in human progress.<\/p>\n<p>Sam Altman said its importance is comparable to that of electricity over a century ago &#8211; warning authorities against resisting adoption.<\/p>\n<p>The chief executive of UK&#8217;s largest chipmaker Arm Holdings, Rene Haas, said he is confident that AI will help us cure cancer in our lifetime.<\/p>\n<p>The &#8216;force for good and evil&#8217;<\/p>\n<p>An &#8220;AI optimist&#8221;, Prof Ward agrees that it can be a powerful tool, like electricity.<\/p>\n<p>There is a significant difference, though.<\/p>\n<p>Unlike electricity, AI agents &#8220;can become autonomous and make decisions by themselves.&#8221;<\/p>\n<p>In many cases, intent won&#8217;t be the problem, but rather the methods an agent chooses to achieve a given goal.<\/p>\n<p>Professor Ward suggests a hypothetical scenario in which a super-intelligent agent is asked to &#8220;reduce global carbon emissions as quickly as possible&#8221;.<\/p>\n<p>&#8220;An innocuous target, isn&#8217;t it? But a really capable system that&#8217;s badly guardrailed might reckon that the most effective measures to reduce global carbon emissions would be to close down industries, ration electricity, or shut down aircraft manufacturing.&#8221;<\/p>\n<p>&#8220;That&#8217;ll reduce global carbon emissions pretty quickly, but that&#8217;s not how we want it to be done&#8221;.<\/p>\n<p>EU policymakers have been scrambling to regulate the rapidly evolving industry.<\/p>\n<p><img decoding=\"async\" alt=\"Hacker, IT and person with code on computer, programming and phishing scam with malware or virus. Hacking, system glitch and cloud computing error in dark room, cyber crime and cybersecurity fail\" src=\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/09\/0024fd57-614.jpg\"\/><br \/>\nIt its latest threat intelligence report, Anthropic described several incidents of &#8220;threat actors&#8221; using Claude<\/p>\n<p>The EU AI Act became law two years ago, though its provisions are taking effect on a phased basis.<\/p>\n<p>New rules on deepfakes and chatbots were introduced in August, obliging companies to make it clear when customers are dealing with a computer instead of a real person.<\/p>\n<p>But preventing AI from being used by &#8220;malicious actors&#8221; is an entirely different challenge.<\/p>\n<p>The military application of AI is a major concern for researchers, with the technology already being used in drones and surveillance.<\/p>\n<p>In its latest threat intelligence report, Anthropic said it &#8220;broke up attempts&#8221; to use its Claude models by &#8220;threat actors&#8221;.<\/p>\n<p>The incidents included attempts by scientists to create biological weapons in one of the regions &#8220;not supported by Anthropic&#8221;.<\/p>\n<p>Those regions include China, Russia and North Korea, among others.<\/p>\n<p>The company also claimed that Russia-linked hackers have been using Claude to carry out espionage campaigns in Ukraine.<\/p>\n<p>It also accused Chinese firms of trying to copy its AI models.<\/p>\n<p>&#8220;Paying attention&#8221; to how the technology is being developed and used is key, according to Prof Ward.<\/p>\n<p>&#8220;It all depends on how we engineer and develop it, and the onus is on us to do it responsibly and to do it well.&#8221;<\/p>\n<p>&#8220;It&#8217;s a force for good. It can also be a force for evil.&#8221;<\/p>\n","protected":false},"excerpt":{"rendered":"Artificial intelligence will have the capability to destroy humanity &#8220;by the end of the decade&#8221;. This alarming claim&hellip;\n","protected":false},"author":2,"featured_media":623978,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[5],"tags":[72,61,60],"class_list":["post-623977","post","type-post","status-publish","format-standard","has-post-thumbnail","category-business","tag-business","tag-ie","tag-ireland"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/623977","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=623977"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/623977\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/623978"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=623977"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=623977"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=623977"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}