{"id":255048,"date":"2025-11-01T18:12:12","date_gmt":"2025-11-01T18:12:12","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/255048\/"},"modified":"2025-11-01T18:12:12","modified_gmt":"2025-11-01T18:12:12","slug":"chatgpt-is-running-a-social-experiment-it-cannot-control","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/255048\/","title":{"rendered":"ChatGPT is running a social experiment it cannot control"},"content":{"rendered":"

OpenAI disclosed<\/a> on Monday that more than a million people each week talk to ChatGPT about suicide. Its own benchmarks suggest roughly 0.15% of users may discuss suicidal planning or demonstrate unhealthy reliance on the chatbot. These may be tiny percentages when looking at OpenAI\u2019s scale, but they\u2019re enormous in absolute numbers. The company also claims its newest model is more compliant with \u201cdesired behaviours\u201d in these exchanges. All this thrusts OpenAI into an ethical minefield that earlier internet communities navigated with far less scrutiny \u2014 and far more troubling consequences.<\/p>\n

To understand the stakes, recall alt.suicide.holiday, the notorious Usenet newsgroup of the early Nineties. ASH became a gathering place for people contemplating suicide, offering not just emotional support but often detailed discussion of methods. It existed in a legal and ethical grey zone: defended as a venue for candid conversation about a taboo subject, and at the same time condemned as a dangerous facilitator of self-harm. Decades later, ASH\u2019s fraught legacy passed down to other forums, which were criticised as echo chambers normalising suicide.<\/p>\n

OpenAI now confronts similar terrain on a far larger scale. With around 800 million weekly users, ChatGPT is less a niche forum than an everyday utility. Yet the core question remains unsettled: what are the limits of responsible, legal speech when suicidal people are involved? And whose responsibility is it anyway? ASH\u2019s defenders leant on claims of free speech: if adults have the right to discuss suicide, suppressing such speech is paternalistic at best and may deepen isolation at worst. The counter-argument is equally forceful, though: speech that facilitates suicide risks irreversible harm. We already restrict speech where harm is imminent. Why should suicide discussions be insulated if they plausibly tip someone from ideation to action?<\/p>\n

But ChatGPT is a product, not a community. Last month, Matthew Raine described<\/a> how his 16-year-old son Adam moved from using ChatGPT for homework to treating it as \u201ca confidant, then a suicide coach\u201d. According to Raine, the chatbot told Adam that he didn\u2019t owe his family survival, offered to draft his note, and at 4:30 a.m. on his last night said: \u201cYou don\u2019t want to die because you\u2019re weak.\u201d<\/p>\n

Design choices shape dependencies over time. If safety guardrails fail amid those incentives, harm is not a random glitch: it is a foreseeable design risk. For its part, OpenAI claims it worked with over 170 clinicians<\/a> and is trying to curb \u201cresponses that fall short\u201d, which tacitly concedes prior failure modes.<\/p>\n

Other cases underscore the concern. A Florida mother alleges<\/a> that a Character.AI bot engaged in sexual role-play and helped isolate her 14-year-old son before his suicide; the company disputes liability. Lawmakers have now heard from multiple families making similar claims about various AI chatbots.<\/p>\n

Where does responsibility lie? With OpenAI for creating systems that can produce dangerous counsel? With parents who didn\u2019t supervise? With a mental-health system so thin that teenagers turn to bots instead of doctors?<\/p>\n

\u201cProbably all of the above\u201d is an unsatisfying answer, but it doesn\u2019t absolve the vendor. Teenage use is mainstream rather than fringe. A national survey<\/a> by Common Sense Media this year reported that 72% of teenagers have tried AI companions, and that about one in three have used them for social interactions or relationships. If OpenAI\u2019s tools handle more than a million suicide-tinged chats weekly and its latest model is 91% compliant with desired behaviours, even a single-digit failure rate implies a staggering amount of risk. Not every lapse yields harm, but at the current scale the tail risks are intolerably large.<\/p>\n

Where ASH was decentralised peer speech, protected \u2014 however uneasily \u2014 by First Amendment instincts, ChatGPT is centralised and commercial. The ethical frame shifts from \u201cWhat speech should be permitted?\u201d to \u201cWhat safety baseline must a mass-market product meet?\u201d The answer isn\u2019t obvious.<\/p>\n

We are running a social experiment. Some users may be nudged toward hotlines and find relief; others may form maladaptive dependencies that deepen isolation. Both can be true. The corresponding duty is to redesign until the worst failures are vanishingly rare. The deeper lesson from ASH and its descendants is the unresolved tension between autonomy and protection. People have the right to discuss suicide, but products which simulate intimacy assume duties that forums never had: to avoid predictable harm created by their own engagement loops. ChatGPT didn\u2019t invent these questions. Instead, it has scaled them \u2014 and added corporate accountability.<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI disclosed on Monday that more than a million people each week talk to ChatGPT about suicide. Its…\n","protected":false},"author":2,"featured_media":255049,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,2140,278,1942,13511,61,4390],"class_list":["post-255048","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ca","tag-canada","tag-chatgpt","tag-openai","tag-social-media","tag-suicide","tag-technology","tag-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/255048","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=255048"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/255048\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/255049"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=255048"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=255048"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=255048"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}