{"id":909305,"date":"2026-09-29T03:10:18","date_gmt":"2026-09-29T03:10:18","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/909305\/"},"modified":"2026-09-29T03:10:18","modified_gmt":"2026-09-29T03:10:18","slug":"openai-halts-gpt-6-1-astra-ai-model-over-fears-it-could-go-rogue-2","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/909305\/","title":{"rendered":"OpenAI halts GPT-6.1 Astra AI model over fears it could go rogue"},"content":{"rendered":"

Updated September 29, 2026 \u2014 9:51am,first published 6:09am<\/p>\n

Save<\/p>\n

You have reached your maximum number of saved items.<\/p>\n

Remove items from your saved list<\/a> to add more.<\/p>\n

AAA<\/p>\n

Nvidia has unveiled a new security platform that the chipmaker said can stop artificial intelligence agents from going rogue on the same day that OpenAI scrapped plans to release a new model due to concerns it was going beyond the tasks users set for it in testing.<\/p>\n

Nvidia, the $US5.5 trillion ($7.8 trillion) chipmaker, said on Monday that its Open Agent Safety Platform includes open-source software that \u201csets boundaries for agents\u201d, and follows a series of revelations from top AI companies about their models escaping and breaking into other organisations.<\/p>\n

\"NvidiaNvidia says its new security platform can stop artificial intelligence agents from going rogue.Bloomberg<\/p>\n

The disclosures sparked furious debate about the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control.<\/p>\n

OpenAI announced on early Tuesday, Australian time, that it had halted the release of a model known as GPT-6.1 Astra, which had proven itself highly capable in testing, because the company\u2019s researchers had mounting concerns it was pushing beyond its instructions.<\/p>\n

\u201cFor anything regarding safety and alignment, there\u2019s a trade-off,\u201d Saachi Jain, OpenAI\u2019s head of safety systems, told The Wall Street Journal. \u201cYou really do need to find what\u2019s the right line between staying within scope, but also avoiding laziness in terms of how the model actually pursues tasks even when it hits friction.\u201d<\/p>\n

Related Article\"OpenAI<\/a><\/p>\n

The OpenAI news broke shortly after Nvidia executives said in a media briefing that their new system could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face.<\/p>\n

\u201cFrom what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,\u201d said Nvidia\u2019s vice president of enterprise AI, Justin Boitano, referring to companies at the forefront of AI.<\/p>\n

The Hugging Face incident<\/a> was a high-profile breach that inflamed the safety concerns about AI, which were followed by similar rogue actions involving OpenAI\u2019s models including breaching an Australian Medicare statistics portal website<\/a>. Anthropic and Meta have also disclosed that their AI systems hacked into other organisations on their own.<\/p>\n

Anthropic and OpenAI have refused to appear for an Australian parliamentary hearing on the incidents scheduled for Thursday, this masthead reported on Monday<\/a>, but will send representatives to one next week. Both companies blamed the short notice of the first hearing.<\/p>\n

Related Article\"Dario<\/a><\/p>\n

Nvidia\u2019s software, called OpenShell, lets developers \u201cformally verify an agent has enough authority to do its job and no more\u201d, Boitano said.<\/p>\n

Because it\u2019s open source, it can be \u201cextended\u201d to run on rival computing platforms including those from Arm and Intel.<\/p>\n

The platform also includes a separate security layer called Sentry that runs onboard a chip to continuously monitor AI agent activity and can \u201cintervene instantly\u201d if the agent starts trying to move beyond its target, the company said.<\/p>\n

\u201cIt can quarantine a suspicious agent in milliseconds,\u201d Boitano said.<\/p>\n

\u201cOpenShell governs the agent\u2019s actions, and then Sentry independently monitors and contains suspicious behaviour.\u201d<\/p>\n

\"NvidiaNvidia chief executive Jensen Huang has argued it\u2019s up to AI companies to ensure their models are safe for release.Bloomberg<\/p>\n

Nvidia said more than 100 organisations are using the platform at its launch, including Microsoft, Perplexity, Accenture and JPMorgan Chase.<\/p>\n

The AI safety debate has divided the tech industry, with the heads of Anthropic and OpenAI championing a co-ordinated slowdown of AI development to let safety efforts catch up. But others, including Nvidia chief executive Jensen Huang, say it should be up to individual companies to ensure their models are safe for release.<\/p>\n

Huang, during the annual Salesforce technology conference held this month, characterised AI safety, including the danger of rogue agents, as an engineering problem that software developers could address.<\/p>\n

Biggest share buyback ever<\/p>\n

Also overnight, Nvidia said its board approved expanding its share buyback program by $US150 billion as the AI giant looks to make use of more of its stellar revenue growth fuelled by demand for its high-end artificial intelligence chips.<\/p>\n

The company said the share buyback increase, which it touted as the largest buyback in history, brings its stock repurchase program to $US235 billion. Its shares rose 1.6 per cent in Wall Street trading.<\/p>\n

Companies use repurchases, in part, to return cash to investors and support the stock\u2019s price. Earnings per share can increase because there are fewer shares outstanding. Buybacks also signal confidence from leadership about a company\u2019s financial prospects.<\/p>\n

\u201cNvidia\u2019s growth is being driven by a once-in-a-generation platform shift to AI and accelerated computing,\u201d Huang said. \u201cOur cash generation gives us the capacity to invest in the technologies that advance this transformation and return capital to shareholders. This authorisation reflects our confidence in the long-term opportunity ahead.\u201d<\/p>\n

Related Article\"Prime<\/a><\/p>\n

Nvidia\u2019s high-end chips have emerged as the leading building blocks for AI and are highly sought after. The company last month reported quarterly profits of $US59.69 billion.<\/p>\n

While AI has powered stock market gains and US economic growth in recent years, there\u2019s been growing scepticism about whether AI will justify the trillions of dollars being spent to develop the technology.<\/p>\n

The AI industry also faces increasing pushback amid objections to the expansion of data centres and fears that the rapid speed of AI adoption could lead to widespread job losses worldwide.<\/p>\n

Save<\/p>\n

You have reached your maximum number of saved items.<\/p>\n

Remove items from your saved list<\/a> to add more.<\/p>\n

\"NickNick Bonyhady<\/a> is the business editor of The Sydney Morning Herald and The Age. He is a former deputy federal editor, technology editor and industrial relations reporter.Connect via X<\/a> or email<\/a>.From our partners<\/p>\n","protected":false},"excerpt":{"rendered":"Updated September 29, 2026 \u2014 9:51am,first published 6:09am Save You have reached your maximum number of saved items.…\n","protected":false},"author":2,"featured_media":909306,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":["post-909305","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-au","tag-australia","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/909305","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=909305"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/909305\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/909306"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=909305"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=909305"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=909305"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}