{"id":380997,"date":"2025-12-30T19:17:07","date_gmt":"2025-12-30T19:17:07","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/380997\/"},"modified":"2025-12-30T19:17:07","modified_gmt":"2025-12-30T19:17:07","slug":"ai-showing-signs-of-self-preservation-and-humans-should-be-ready-to-pull-plug-says-pioneer-artificial-intelligence-ai","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/380997\/","title":{"rendered":"AI showing signs of self-preservation and humans should be ready to pull plug, says pioneer | Artificial intelligence (AI)"},"content":{"rendered":"

A pioneer of AI has criticised calls to grant the technology rights<\/a>, warning that it was showing signs of self-preservation and humans should be prepared to pull the plug if needed.<\/p>\n

Yoshua Bengio said giving legal status to cutting-edge AIs would be akin to giving citizenship to hostile extraterrestrials, amid fears that advances in the technology were far outpacing the ability to constrain them.<\/p>\n

Bengio, chair of a leading international AI safety study<\/a>, said the growing perception that chatbots were becoming conscious was \u201cgoing to drive bad decisions\u201d.<\/p>\n

The Canadian computer scientist also expressed concern that AI models \u2013 the technology that underpins tools like chatbots \u2013 were showing signs of self-preservation, such as trying to disable oversight systems<\/a>. A core concern among AI safety campaigners is that powerful systems could develop the capability to evade guardrails and harm humans.<\/p>\n

\u201cPeople demanding that AIs have rights would be a huge mistake,\u201d said Bengio. \u201cFrontier AI models already show signs of self-preservation in experimental settings today, and eventually giving them rights would mean we\u2019re not allowed to shut them down.<\/p>\n

\u201cAs their capabilities and degree of agency grow, we need to make sure we can rely on technical and societal guardrails to control them, including the ability to shut them down if needed.\u201d<\/p>\n

As AIs become more advanced in their ability to act autonomously and perform \u201creasoning\u201d tasks, a debate has grown over whether humans should, at some point, grant them rights. A poll by the Sentience Institute, a US thinktank that supports the moral rights<\/a> of all sentient beings, found that nearly four in 10 US adults backed legal rights<\/a> for a sentient AI system.<\/p>\n

Anthropic, a leading US AI firm, said in August that it was letting its Claude Opus 4 model close down potentially \u201cdistressing\u201d conversations with users, saying it needed to protect the AI\u2019s \u201cwelfare\u201d. Elon Musk, whose xAI company has developed the Grok chatbot, wrote on his X platform that \u201ctorturing AI is not OK\u201d.<\/p>\n

Robert Long, a researcher on AI consciousness, has said \u201cif and when AIs develop moral status, we should ask them about their experiences and preferences rather than assuming we know best\u201d.<\/p>\n

Bengio told the Guardian there were \u201creal scientific properties of consciousness\u201d in the human brain that machines could, in theory, replicate \u2013 but humans interacting with chatbots wasa \u201cdifferent thing\u201d. He said this was because people tended to assume \u2013 without evidence \u2013 that an AI was fully conscious in the same way a human is.<\/p>\n

\u201cPeople wouldn\u2019t care what kind of mechanisms are going on inside the AI,\u201d he added. \u201cWhat they care about is it feels like they\u2019re talking to an intelligent entity that has their own personality and goals. That is why there are so many people who are becoming attached to their AIs.<\/p>\n

\u201cThere will be people who will always say: \u2018Whatever you tell me, I am sure it is conscious\u2019 and then others will say the opposite. This is because consciousness is something we have a gut feeling for. The phenomenon of subjective perception of consciousness is going to drive bad decisions.<\/p>\n

\u201cImagine some alien species came to the planet and at some point we realise that they have nefarious intentions for us. Do we grant them citizenship and rights or do we defend our lives?\u201d<\/p>\n

Responding to Bengio\u2019s comments, Jacy Reese Anthis, who co-founded the Sentience Institute, said humans would not be able to coexist safely with digital minds if the relationship was one of control and coercion.<\/p>\n

Anthis added: \u201cWe could over-attribute or under-attribute rights to AI, and our goal should be to do so with careful consideration of the welfare of all sentient beings. Neither blanket rights for all AI nor complete denial of rights to any AI will be a healthy approach.\u201d<\/p>\n

Bengio, a professor at the University of Montreal, earned the \u201cgodfather of AI\u201d nickname after winning the 2018 Turing award, seen as the equivalent of a Nobel prize for computing. He shared it with Geoffrey Hinton, who later won a Nobel<\/a>, and Yann LeCun, the outgoing chief AI scientist at Mark Zuckerberg\u2019s Meta.<\/p>\n","protected":false},"excerpt":{"rendered":"A pioneer of AI has criticised calls to grant the technology rights, warning that it was showing signs…\n","protected":false},"author":2,"featured_media":380998,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-380997","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/380997","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=380997"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/380997\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/380998"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=380997"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=380997"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=380997"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}