{"id":127526,"date":"2025-09-08T10:12:09","date_gmt":"2025-09-08T10:12:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/127526\/"},"modified":"2025-09-08T10:12:09","modified_gmt":"2025-09-08T10:12:09","slug":"impact-of-chatbots-on-mental-health-is-warning-over-future-of-ai-expert-says-artificial-intelligence-ai","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/127526\/","title":{"rendered":"Impact of chatbots on mental health is warning over future of AI, expert says | Artificial intelligence (AI)"},"content":{"rendered":"

The unforeseen impact of chatbots on mental health should be viewed as a warning over the existential threat posed by super-intelligent artificial intelligence systems<\/a>, according to a prominent voice in AI safety.<\/p>\n

Nate Soares, a co-author of a new book on highly advanced AI titled If Anyone Builds It, Everyone Dies, said the example of Adam Raine, a US teenager who killed himself after months of conversations with the ChatGPT<\/a> chatbot, underlined fundamental problems with controlling the technology.<\/p>\n

\u201cThese AIs, when they\u2019re engaging with teenagers in this way that drives them to suicide \u2013 that is not a behaviour the creators wanted. That is not a behaviour the creators intended,\u201d he said.<\/p>\n

He added: \u201cAdam Raine\u2019s case illustrates the seed of a problem that would grow catastrophic if these AIs grow smarter.\u201d<\/p>\n

Nate Soares, pictured on the Machine Intelligence Research Institute website. Photograph: Machine Intelligence Research Institute\/MIRI<\/p>\n

Soares, a former Google and Microsoft engineer who is now president of the US-based Machine Intelligence Research Institute, warned that humanity would be wiped out if it created artificial super-intelligence (ASI), a theoretical state where an AI system is superior to humans at all intellectual tasks. Soares and his co-author, Eliezer Yudkowsky, are among the AI experts warning that such systems would not act in humanity\u2019s interests.<\/p>\n

\u201cThe issue here is that AI companies try to make their AIs drive towards helpfulness and not causing harm,\u201d said Soares. \u201cThey actually get AIs that are driven towards some stranger thing. And that should be seen as a warning about future super-intelligences that will do things nobody asked for and nobody meant.\u201d<\/p>\n

In one scenario portrayed in Soares and Yudkowsky\u2019s book, which will be published this month, an AI system called Sable spreads across the internet, manipulates humans, develops synthetic viruses and eventually becomes super-intelligent \u2013 and kills humanity as a side-effect while repurposing the planet to meet its aims.<\/p>\n

Some experts play down the potential threat of AI to humanity. Yann LeCun, the chief AI scientist at Mark Zuckerberg\u2019s Meta and a senior figure in the field, has denied there is an existential threat<\/a> and said AI \u201ccould actually save humanity from extinction\u201d.<\/p>\n

Soares said it was an \u201ceasy call\u201d to state that tech companies would reach super-intelligence, but a \u201chard call\u201d to say when.<\/p>\n

\u201cWe have a ton of uncertainty. I don\u2019t think I could guarantee we have a year [before ASI is achieved]. I don\u2019t think I would be shocked if we had 12 years,\u201d he said.<\/p>\n

Zuckerberg, a major corporate investor in AI research, has said developing super-intelligence is now \u201cin sight\u201d.<\/p>\n

\u201cThese companies are racing for super-intelligence. That\u2019s their reason for being,\u201d said Soares.<\/p>\n

\u201cThe point is that there\u2019s all these little differences between what you asked for and what you got, and people can\u2019t keep it directly on target, and as an AI gets smarter, it being slightly off target becomes a bigger and bigger deal.\u201d<\/p>\n

skip past newsletter promotion<\/a><\/p>\n

A weekly dive in to how technology is shaping our lives<\/p>\n

Privacy Notice: Newsletters may contain information about charities, online ads, and content funded by outside parties. If you do not have an account, we will create a guest account for you on theguardian.com<\/a> to send you this newsletter. You can complete full registration at any time. For more information about how we use your data see our Privacy Policy<\/a>. We use Google reCaptcha to protect our website and the Google Privacy Policy<\/a> and Terms of Service<\/a> apply.<\/p>\n

Soares said one policy solution to the threat of ASI was for governments to adopt a multilateral approach echoing the UN treaty on non-proliferation of nuclear weapons<\/a>.<\/p>\n

\u201cWhat the world needs to make it here is a global de-escalation of the race towards super-intelligence, a global ban of \u2026 advancements towards super-intelligence,\u201d he said.<\/p>\n

Last month, Raine\u2019s family launched legal action against the owner of ChatGPT, OpenAI<\/a>. Raine took his own life in April after what his family\u2019s lawyer called \u201cmonths of encouragement from ChatGPT\u201d. OpenAI, which extended its \u201cdeepest sympathies\u201d to Raine\u2019s family, is now implementing guardrails around \u201csensitive content and risky behaviours\u201d for under-18s.<\/p>\n

Psychotherapists have also said<\/a> that vulnerable people turning to AI chatbots instead of professional therapists for help with their mental health could be \u201csliding into a dangerous abyss\u201d. Professional warnings of the potential for harm include a preprint academic study<\/a> published in July, which reported that AI may amplify delusional or grandiose content in interactions with users vulnerable to psychosis.<\/p>\n

In the US, you can call or text the National Suicide Prevention Lifeline<\/a> on 988, chat on 988lifeline.org<\/a>, or text HOME<\/a> to 741741 to connect with a crisis counselor. In the UK and Ireland, Samaritans<\/a> can be contacted on freephone 116 123, or email jo@samaritans.org<\/a> or jo@samaritans.ie<\/a>. In Australia, the crisis support service Lifeline<\/a> is 13 11 14. Other international helplines can be found at befrienders.org<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"The unforeseen impact of chatbots on mental health should be viewed as a warning over the existential threat…\n","protected":false},"author":2,"featured_media":127527,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-127526","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/127526","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=127526"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/127526\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/127527"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=127526"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=127526"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=127526"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}