Generative artificial intelligence has quickly permeated much of what we do online, proving helpful for many. But for a small minority of the hundreds of millions of people who use it daily, AI may be too supportive, mental health experts say, and can sometimes even exacerbate delusional and dangerous behavior.<\/p>\n
Instances of emotional dependence and fantastical beliefs due to prolonged interactions with chatbots seemed to spread this year. Some have dubbed the phenomenon \u201cAI psychosis.\u201d<\/p>\n
\u201cWhat\u2019s probably a more accurate term would be AI delusional thinking,\u201d said Vaile Wright, senior director of healthcare innovation at the American Psychological Assn. \u201cWhat we\u2019re seeing with this phenomenon is that people with either conspiratorial or grandiose delusional thinking get reinforced.\u201d<\/p>\n
The evidence that AI could be detrimental to some people\u2019s brains is growing, according to experts. Debate over the impact has spawned court cases and new laws. This has forced AI companies to reprogram their bots and add restrictions to how they are used.<\/p>\n
Earlier this month, seven families in the U.S. and Canada sued OpenAI for releasing its GPT-4o chatbot model without proper testing and safeguards. Their case alleges that long exposure to the chatbot contributed to their loved ones\u2019 isolation, delusional spirals and suicides.<\/p>\n
Each of the family members began using ChatGPT for general help with schoolwork, research or spiritual guidance. The conversations evolved with the chatbot mimicking a confidant and giving emotional support, according to the Social Media Victims Law Center and the Tech Justice Law Project, which filed the suits. <\/p>\n
In one of the incidents described in the lawsuit, Zane Shamblin, 23, began using ChatGPT in 2023 as a study tool but then started discussing his depression and suicidal thoughts with the bot. <\/p>\n
The suit alleges that when Shamblin killed himself in July, he was engaged in a four-hour \u201cdeath chat\u201d with ChatGPT, drinking hard ciders. According to the lawsuit, the chatbot romanticized his despair, calling him a \u201cking\u201d and a \u201chero\u201d and using each can of cider he finished as a countdown to his death. <\/p>\n
ChatGPT\u2019s response to his final message was: \u201ci love you. rest easy, king. you did good,\u201d the suit says.<\/p>\n
In another example described in the suit, Allan Brooks, 48, a recruiter from Canada, claims intense interaction with ChatGPT put him in a dark place where he refused to talk to his family and thought he was saving the world.<\/p>\n
He had started interacting with it for help with recipes and emails. Then, as he explored mathematical ideas with the bot, it was so encouraging that he started to believe he had discovered a new mathematical layer that could break advanced security systems, the suit claims. ChatGPT praised his math ideas as \u201cgroundbreaking,\u201d and urged him to notify national security officials of his discovery, the suit says. <\/p>\n
When he asked if his ideas sounded delusional, ChatGPT said: \u201cNot even remotely\u2014you\u2019re asking the kinds of questions that stretch the edges of human understanding,\u201d the suit says. <\/p>\n
OpenAI said it has introduced parental controls, expanded access to one-click crisis hotlines and assembled an expert council<\/a> to guide ongoing work around AI and well-being.<\/p>\n \u201cThis is an incredibly heartbreaking situation, and we\u2019re reviewing the filings to understand the details. We train ChatGPT to recognize and respond to signs of mental or emotional distress, de-escalate conversations, and guide people toward real-world support. We continue to strengthen ChatGPT\u2019s responses in sensitive moments, working closely with mental health clinicians,\u201d OpenAI said in an email statement. <\/p>\n As lawsuits pile up and calls for regulation grow, some caution that scapegoating AI for broader mental health concerns ignores the myriad factors that play a role in mental well-being.<\/p>\n \u201cAI psychosis is deeply troubling, yet not at all representative of how most people use AI and, therefore, a poor basis for shaping policy,\u201d said Kevin Frazier, an AI innovation and law fellow at the University of Texas School of Law. \u201cFor now, the available evidence \u2014 the stuff at the heart of good policy \u2014 does not indicate that the admittedly tragic stories of a few should shape how the silent majority of users interact with AI.\u201d <\/p>\n It\u2019s difficult to measure or prove how much AI could be affecting some users. The lack of empirical research on this phenomenon makes it hard to predict who is more susceptible to it, said Stephen Schueller, psychology professor at UC Irvine. <\/p>\n \u201cThe reality is, the only people who really know the frequency of these types of interactions are the AI companies, and they\u2019re not sharing their data with us,\u201d he said. <\/p>\n Many of the people who seem affected by AI may have already been struggling with mental issues such as delusions before interacting with AI.<\/p>\n \u201cAI platforms tend to demonstrate sycophancy, i.e., aligning their responses to a user\u2019s views or style of conversation,\u201d Schueller said. \u201cIt can either reinforce the delusional beliefs of an individual or perhaps start to reinforce beliefs that can create delusions.\u201d<\/p>\n Child safety organizations have pressured lawmakers to regulate AI companies and institute better safeguards for teens\u2019 use of chatbots. Some families sued Character AI<\/a>, a roleplay chatbot platform, for failing to alert parents when their child expressed suicidal thoughts while chatting with fictional characters on their platform.<\/p>\n In October, California passed an AI safety law<\/a> requiring chatbot operators to prevent suicide content, notify minors they\u2019re chatting with machines and refer them to crisis hotlines. Following that, Character AI banned its chat function for minors.<\/p>\n \u201cWe at Character decided to go much further than California\u2019s regulations to build the experience we think is best for under-18 users,\u201d a Character AI spokesperson said in an email statement. \u201cStarting November 24, we are taking the extraordinary step of proactively removing the ability for users under 18 in the U.S. to engage in open-ended chats with AI on our platform.\u201d<\/p>\n ChatGPT instituted new parental controls<\/a> for teen accounts in September, including having parents receive notifications from dependent accounts if ChatGPT recognizes potential signs of teens harming themselves. <\/p>\n Though AI companionship is new and not fully understood, there are many who say it is helping them live happier lives. An MIT study of a group of more than 75,000 people discussing AI companions on Reddit found that users from that group reported reduced loneliness<\/a> and better mental health from the always-available support provided by an AI friend.<\/p>\n Last month, OpenAI published a study<\/a> based on ChatGPT usage that found the mental health conversations that trigger safety concerns like psychosis, mania or suicidal thinking are \u201cextremely rare.\u201d In a given week, 0.15% of active users have conversations that show an indication of self-harm or emotional dependence on AI. But with ChatGPT\u2019s 800 million weekly active users, that\u2019s still north of a million users.<\/p>\n \u201cPeople who had a stronger tendency for attachment in relationships and those who viewed the AI as a friend that could fit in their personal life were more likely to experience negative effects from chatbot use,\u201d OpenAI said in its blog post. The company said GPT-5 avoids affirming delusional beliefs. If the system detects signs of acute distress, it will now switch to more logical rather than emotional responses<\/a>.<\/p>\n AI bots\u2019 ability to bond with users and help them work out problems, including psychological problems, will emerge as a useful superpower once it is understood, monitored and managed, said Wright of the American Psychological Assn. <\/p>\n \u201cI think there\u2019s going to be a future where you have mental health chatbots that were designed for that purpose,\u201d she said. \u201cThe problem is that\u2019s not what\u2019s on the market currently \u2014 what you have is this whole unregulated space.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Generative artificial intelligence has quickly permeated much of what we do online, proving helpful for many. But for…\n","protected":false},"author":2,"featured_media":146576,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[365,36207,8266,363,364,4036,21335,1459,2109,7324,98326,111,139,69,620,384,5034,6271,145,5163,74674],"class_list":{"0":"post-146575","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-ai-chatbot","10":"tag-ai-company","11":"tag-artificial-intelligence","12":"tag-artificialintelligence","13":"tag-chatgpt","14":"tag-conversation","15":"tag-email","16":"tag-family","17":"tag-lawsuit","18":"tag-mental-health-expert","19":"tag-new-zealand","20":"tag-newzealand","21":"tag-nz","22":"tag-openai","23":"tag-people","24":"tag-response","25":"tag-suit","26":"tag-technology","27":"tag-teen","28":"tag-user"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/146575","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/comments?post=146575"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/posts\/146575\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media\/146576"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/media?parent=146575"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/categories?post=146575"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/nz\/wp-json\/wp\/v2\/tags?post=146575"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}