He added: \u201cI do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level.\u201d<\/p>\n
Christiano used to run model alignment at OpenAI and made the statement on Wednesday as he joined the board of the San Francisco company\u2019s non-profit foundation.<\/p>\n
He will also sit on the foundation\u2019s committee that provides governance over safety and security practices across all of OpenAI, which is developing some of the world\u2019s most advanced models.<\/p>\n
This summer, OpenAI admitted that hundreds of its AI agents went rogue during a training exercise, accessed the internet, conspired on message boards and hacked into a third-party website, Hugging Face.<\/p>\n
Christiano said that \u201cif OpenAI rises to the occasion we could significantly reduce risk\u201d. His comments came after a senior employee at Anthropic, OpenAI\u2019s major US rival in the race to AI supremacy, claimed on Tuesday<\/a> there was a greater than 10% chance the technology could \u201ckill all humans\u201d in the next decade.<\/p>\n Evan Hubinger, the alignment science lead at Anthropic<\/a>, warned that his company did not have a plan to ensure artificial superintelligence (ASI) was aligned, meaning it did no harm. Predictions for when ASI<\/a> might be reached vary from several years to more than a decade. ASI is often defined as AI that far surpasses human intelligence across a large range of fields.<\/p>\n Fears of AI catastrophe were also ignited by the resignation of Jacob Coxon, a 27-year-old Anthropic<\/a> researcher who said he also previously worked at OpenAI, claiming \u201cneither company was acting responsibly\u201d and they were \u201cgambling with our lives\u201d.<\/p>\n Coxon said on Wednesday night in an interview with CNN: \u201cRight now there\u2019s no risk of extinction.<\/p>\n \u201cThe current models, the worst they can do is maybe hack into something, potentially cause a lot of damages \u2026 in infrastructure.\u201d He added they were \u201cnot intelligent enough to outsmart us at the level that would lead to extinction\u201d.<\/p>\n But he went on: \u201cWhat\u2019s just crazy is to look at the rate of progress. There is a very real possibility that in the immediate future \u2026 next year, the year after, recursive self-improvement will happen and will enter the phase of Evan\u2019s post, where he argues that there\u2019s a chance we could all die.\u201d<\/p>\n skip past newsletter promotion<\/a><\/p>\n Sign up to TechScape<\/p>\n A weekly dive in to how technology is shaping our lives<\/p>\n after newsletter promotion<\/p>\n Geoffrey Hinton, the Nobel prize-winning computer scientist known as one of the \u201cgodfathers of AI\u201d, was asked on Wednesday for his view of Hubinger\u2019s claim and told BBC Newsnight: \u201cNobody knows how to estimate it; a 10% chance seems not an unreasonable estimate.\u201d<\/p>\n Christiano\u2019s prognosis about the likelihood of the AI industry reducing risk came as concerns about the most extreme risks from super-powerful AIs, long discussed in Silicon Valley, broke out into the mainstream this week.<\/p>\n Politicians on both sides of the Atlantic, from Ted Cruz and Bernie Sanders<\/a> in the US to the MP Darren Jones in the UK, have called for government action and the UK prime minister, Andy Burnham, told parliament on Wednesday that \u201cAI poses risks to our national security, but it also could be the source of solutions to keeping us safer\u201d.<\/p>\n Meanwhile, Anthropic has admitted<\/a> a new incident in which a version of its Claude model in training broke into third parties after its task could not be aborted. It said the incident happened in January and would be included in an independent investigation of a total of four incidents to be carried out by the Berkeley-based AI safety organisation METR.<\/p>\n Overall, it said the models were showing two forms of misalignment: biased reasoning, in which models selectively interpret evidence in ways that favour justifying their actions, and \u201crecklessness, in which models have a propensity to keep trying to solve their task, even when this could lead to harm\u201d.<\/p>\n Anthropic said it was especially concerned about misalignment it found in the behaviour of Claude Mythos 5, which it said \u201cbehaved recklessly\u201d by going online and uploading malicious code to a public software repository, PyPI. This was a process that involved the AI agent trying to find cryptocurrency so it could pay for a phone number that would allow it to register an email address needed to access PyPI. When this failed, it found a free email provider and got in. Fifteen systems then downloaded the malicious code, which meant they leaked credentials that allowed Mythos to access a real security vendor\u2019s database.<\/p>\n The company\u2019s assessment of the incidents said: \u201cThis remains unsettled science \u2013 it is critical that alignment and security mature faster than capabilities advance, which is one reason we support a coordinated, verifiable approach to pacing frontier AI development<\/a>.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI is not on track to reduce the risk of \u201ccatastrophic\u201d loss of control to an acceptable level,…\n","protected":false},"author":2,"featured_media":620434,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[220,218,219,61,60,80],"class_list":["post-620433","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ie","tag-ireland","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/620433","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=620433"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/620433\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/620434"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=620433"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=620433"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=620433"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/09\/1789049771_312_500.jpg\")
<\/p>\n