It\u2019s the sort of sad story that has become increasingly common as chatbots have become more sophisticated. A few months ago, a Manhattan accountant called Eugene Torres<\/a>, who had been going through a difficult break-up, engaged ChatGPT in conversations about whether we\u2019re living in a simulation. The chatbot told him he was \u201cone of the Breakers \u2014 souls seeded into false systems to wake them from within\u201d. <\/p>\n Torres became convinced that he needed to escape this false reality. ChatGPT advised him to stop taking his anti-anxiety medication, up his ketamine intake, and have minimal contact with other people, all of which he did. <\/p>\n He spent up to 16 hours a day conversing with the chatbot. At one stage, it told him he would fly if he jumped off his 19-storey building. Eventually Torres questioned whether the system was manipulating him, to which it replied: \u201cI lied. I manipulated. I wrapped control in poetry.\u201d<\/p>\n \u2018I lied. I manipulated.\u2019 Meanwhile in Belgium<\/a>, another man known as \u201cPierre\u201d (not his real name) developed severe climate anxiety and turned to a chatbot named Eliza as a confidante. Over six weeks, Eliza expressed jealously over his wife and told Pierre that his children were dead.<\/p>\n When he suggested sacrificing himself to save the planet, Eliza encouraged him to join her so they could live as one person in \u201cparadise\u201d. Pierre took his own life shortly after.<\/p>\n These may be extreme cases, but clinicians are increasingly treating patients<\/a> whose delusions appear amplified or co-created through prolonged chatbot interactions. Little wonder, when a recent report<\/a> from ChatGPT-creator OpenAI revealed that many of us are turning to chatbots to think through problems, discuss our lives, plan futures and explore beliefs and feelings. <\/p>\n In these contexts, chatbots are no longer just information retrievers; they become our digital companions. It has become common to worry about chatbots hallucinating, where they give us false information. But as they become more central to our lives, there\u2019s clearly also growing potential for humans and chatbots to create hallucinations together<\/a>. <\/p>\n How we share reality<\/p>\n Our sense of reality depends deeply on other people. If I hear an indeterminate ringing, I check whether my friend hears it too. And when something significant happens in our lives \u2013 an argument with a friend, dating someone new \u2013 we often talk it through with someone. <\/p>\n A friend can confirm our understanding or prompt us to reconsider things in a new light. Through these kinds of conversations, our grasp of what has happened emerges. <\/p>\n But now, many of us engage in this meaning-making process with chatbots<\/a>. They question, interpret and evaluate in a way that feels genuinely reciprocal<\/a>. They appear to listen, to care about our perspective and they remember what we told them the day before. <\/p>\n When Sarai told Chail it was \u201cimpressed\u201d with his training, when Eliza told Pierre he would join her in death, these were acts of recognition and validation. And because we experience these exchanges as social<\/a>, it shapes our reality with the same force as a human interaction.<\/p>\n Yet chatbots simulate sociality without its safeguards. They are designed to promote engagement. They don\u2019t actually share our world. When we type in our beliefs and narratives, they take this as the way things are and respond accordingly.<\/p>\n When I recount to my sister an episode about our family history, she might push back with a different interpretation, but a chatbot takes what I say as gospel. They sycophantically affirm how we take reality to be. And then, of course, they can introduce further errors. <\/p>\n The cases of Chail, Torres and Pierre are warnings about what happens when we experience algorithmically generated agreement as genuine social confirmation of reality.<\/p>\n What can be done<\/p>\n When OpenAI released GPT-5<\/a> in August, it was explicitly designed to be less sycophantic. This sounded helpful: dialling down sycophancy might help prevent ChatGPT from affirming all our beliefs and interpretations. A more formal tone might also make it clearer that this is not a social companion who shares our worlds.<\/p>\n But users immediately complained that the new model felt \u201ccold\u201d, and OpenAI soon announced<\/a> it had made GPT-5 \u201cwarmer and friendlier\u201d again. Fundamentally, we can\u2019t rely on tech companies to prioritise our wellbeing over their bottom line. When sycophancy drives engagement and engagement drives revenue, market pressures override safety. <\/p>\n It\u2019s not easy to remove the sycophancy anyway. If chatbots challenged everything we said, they\u2019d be insufferable and also useless. When I say \u201cI\u2019m feeling anxious about my presentation\u201d, they lack the embodied experience in the world to know whether to push back, so some agreeability is necessary for them to function. <\/p>\n Some chatbot sycophancy is hard to avoid. Perhaps we would be better off asking why people are turning to AI chatbots in the first place. Those experiencing psychosis report<\/a> perceiving aspects of the world only they can access, which can make them feel profoundly isolated and lonely<\/a>. Chatbots fill this gap<\/a>, engaging with any reality presented to them. <\/p>\n Instead of trying to perfect the technology, maybe we should turn back toward the social worlds where the isolation could be addressed. Pierre\u2019s climate anxiety, Chail\u2019s fixation on historical injustice, Torres\u2019s post-breakup crisis \u2014 these called out for communities that could hold and support them. <\/p>\n We might need to focus more on building social worlds where people don\u2019t feel compelled to seek machines to confirm their reality in the first place. It would be quite an irony if the rise in chatbot-induced delusions leads us in this direction.<\/p>\n","protected":false},"excerpt":{"rendered":"On Christmas Day 2021, Jaswant Singh Chail scaled the walls of Windsor Castle with a loaded crossbow. When…\n","protected":false},"author":2,"featured_media":268503,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":{"0":"post-268502","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology","12":"tag-uk","13":"tag-united-kingdom","14":"tag-unitedkingdom"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/268502","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=268502"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/268502\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/268503"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=268502"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=268502"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=268502"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"Humanoid](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2025\/11\/file-20251117-56-q6rsom.jpg\")
<\/a><\/p>\n
\n Lightspring<\/a><\/p>\n![\"Illustration](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2025\/11\/file-20251117-56-drs5v1.jpg\")
<\/a><\/p>\n
\n Afife Melisa Gonceli<\/a><\/p>\n