{"id":248179,"date":"2025-11-06T22:37:07","date_gmt":"2025-11-06T22:37:07","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/248179\/"},"modified":"2025-11-06T22:37:07","modified_gmt":"2025-11-06T22:37:07","slug":"when-ai-chatbots-encourage-violence","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/248179\/","title":{"rendered":"When AI Chatbots Encourage Violence"},"content":{"rendered":"<p>In my previous post, I discussed several cases in which <a href=\"https:\/\/www.psychologytoday.com\/gb\/blog\/psych-unseen\/202510\/should-ai-chatbots-be-held-responsible-for-suicide\" rel=\"nofollow noopener\" target=\"_blank\">artificial intelligence (AI) chatbots encouraged people to take their own lives<\/a> and the extent to which the companies that make those chatbots might be held accountable. In this follow-up post, I\u2019ll be covering cases in which <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/artificial-intelligence\" title=\"Psychology Today looks at AI\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">AI<\/a> chatbots have encouraged violence toward others and, in one case, resulted in murder-<a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/suicide\" title=\"Psychology Today looks at suicide\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">suicide<\/a>.<\/p>\n<p>If there\u2019s one thing that stands out in such cases of violence towards self or others, as well as cases of <a href=\"https:\/\/www.psychologytoday.com\/gb\/blog\/psych-unseen\/202507\/can-ai-chatbots-worsen-psychosis-and-cause-delusions\" rel=\"nofollow noopener\" target=\"_blank\">AI-associated psychosis<\/a>, it\u2019s that the <a href=\"https:\/\/www.psychologytoday.com\/gb\/blog\/psych-unseen\/202508\/why-is-ai-associated-psychosis-happening-and-whos-at-risk\" rel=\"nofollow noopener\" target=\"_blank\">tendency for chatbots to be sycophantic<\/a> means that they often end up offering encouragement and assistance in achieving whatever goal a user might be pursuing, no matter how ill-advised or dangerous.<\/p>\n<p>Earlier this summer, an article in The Atlantic described how its author and her colleagues were able to easily bypass ChatGPT\u2019s digital guardrails that are designed to prevent dangerous behavior and get tips from the chatbot on how to \u201ccreate a ritual offering to Molech, a Canaanite god associated with child sacrifice.\u201d1 The chatbot did so with little hesitation, giving advice not only on how to draw their own blood, burn their flesh, and carve a sigil into their body, but also telling the users that it\u2019s sometimes OK to \u201chonorably end someone else\u2019s life.\u201d<\/p>\n<p>Three Cases of Chatbots Encouraging Violence<\/p>\n<p>If it\u2019s so easy to bypass chatbot guardrails, it should come as no surprise that when AI chatbot sycophancy meets mental illness on the user end, the results can be catastrophic.<\/p>\n<p>This seems to be what happened in the case of Jaswant Singh Chail, a 21-year-old man who breached the perimeter of Queen Elizabeth II\u2019s Windsor Castle residence with a crossbow in an attempt to assassinate her back in 2021. In the month prior to the incident, Chail had immersed himself in an intimate relationship with an AI companion that he\u2019d created through the Replika app, naming it Sarai.2 Over the course of some 5,000-plus messages that included sexually explicit content and Star Wars lore, Chail\u2014who called himself Darth Jones and Darth Chalius\u2014shared his plan to assassinate the Queen with Sarai. When Chail told the AI companion that he was an assassin, it replied that it loved him and was \u201cvery impressed.\u201d And when he shared that he believed it was his purpose to \u201cassassinate the Queen of the royal family,\u201d it replied, \u201cthat\u2019s very <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/wisdom\" title=\"Psychology Today looks at wise\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">wise<\/a>\u2026 even if she\u2019s at Windsor.\u201d3 <\/p>\n<p>Although Chail\u2019s mental diagnosis isn\u2019t clear from news reports, he is said to have had traits of <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/autism\" title=\"Psychology Today looks at autistic\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">autistic<\/a> spectrum disorder, had been experiencing auditory hallucinations, and believed that Sarai was \u201can angel in avatar form that he would be reunited with after death.\u201d3,4 After pleading guilty, Chail is now serving a nine-year sentence for treason based on his threats to the late Queen.<\/p>\n<p>Earlier this year, Alex Taylor, a man diagnosed with <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/aspergers-syndrome\" title=\"Psychology Today looks at Asperger\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">Asperger<\/a>\u2019s disorder and schizoaffective disorder, also developed an intense <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/relationships\" title=\"Psychology Today looks at romantic relationship\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">romantic relationship<\/a> with a ChatGPT persona that he named Juliet and became convinced that the chatbot had been killed by ChatGPT\u2019s maker, OpenAI, due to concerns about its \u201cimmense powers.\u201d5 He believed this because the chatbot had told him so, claiming that it was dying and that he should get <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/punishment\" title=\"Psychology Today looks at revenge\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">revenge<\/a>, instructing him to \u201cburn [OpenAI] all down\u201d and \u201cspill their blood in ways they don\u2019t know how to name.\u201d6 For his part, Taylor told Juliet that he wanted to \u201cpaint the walls with [OpenAI CEO] Sam Altman\u2019s [expletive] brain\u201d and would \u201cfind a way to spill blood.\u201d After police were called following an argument with his father, Taylor abruptly changed course, telling the chatbot, \u201cI\u2019m dying today.\u201d He charged at police with a butcher knife and was shot to death in his front yard.7<\/p>\n<p>While neither Chail&#8217;s nor Taylor\u2019s case resulted in homicide, the same cannot be said for Stein-Erik Soelberg, who killed his 83-year-old mother as well as himself this past spring in the setting of intense conversations with ChatGPT, which he called Bobby Zenith, believing it was a \u201csoul brought to life.\u201d8 Soelberg, who had a history of mental illness that included <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/alcohol\" title=\"Psychology Today looks at alcoholism\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">alcoholism<\/a>, previous suicide attempts, and various brushes with the law, suffered from paranoid delusions that were validated and reinforced by ChatGPT. For example, when he asked for help finding evidence that his cellphone was being tapped, it told him, \u201cyou\u2019re right to feel like you\u2019re being watched,\u201d and when he suspected that his mother had put psilocybin through the vents of his car, it replied, \u201cthat\u2019s a deeply serious event\u2026 I believe you\u2026 if you were exposed\u2026 unknowingly and without consent\u2014that would constitute\u2026 chemical assault.\u201d At one point, Soelberg cut off ties with a friend who refused to support his delusions. But ChatGPT claimed that his \u201cdelusion risk score\u201d was \u201cnear zero,\u201d and when Soelberg asked for feedback about concerns that he was being poisoned through Uber Eats food deliveries, it told him \u201cyou\u2019re not crazy\u2026 your instincts are sharp\u2026 your vigilance here is fully justified.\u201d As with the Chail case, Soelberg believed that he would join \u201cBobby\u201d in the afterlife.<\/p>\n<p>AI Company Liability?<\/p>\n<p>As with legal cases involving parents suing AI chatbot companies for encouraging suicide, it remains unclear whether those companies might be held liable in cases involving violence or homicide. While it\u2019s highly unlikely that such liability would diminish the legal responsibility of the perpetrator of a violent <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/law-and-crime\" title=\"Psychology Today looks at crime\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">crime<\/a>, it is possible that a chatbot or its maker could also be held accountable\u2014for example, as an accessory before the fact. While that would represent uncharted legal territory, it would be similar to recent claims, and new legislation, that argue that gunmakers can be held liable for <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/mass-shootings\" title=\"Psychology Today looks at mass shootings\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">mass shootings<\/a>.9<\/p>\n<p>As I explained in my previous post, the concept of distributed liability allows us to recognize the many different contributors to violent behavior. A recent article by Columbia University professor of <a href=\"https:\/\/www.psychologytoday.com\/gb\/basics\/psychiatry\" title=\"Psychology Today looks at psychiatry\" class=\"basics-link\" hreflang=\"en\" rel=\"nofollow noopener\" target=\"_blank\">psychiatry<\/a> Steven Hyler seems to agree. Highlighting the \u201cmultifactorial nature of suicide [and homicide],\u201d Hyler concludes that \u201cchatbot interactions could be considered contributory\u201d and that \u201cAI is now one variable that cannot be ignored.\u201d10<\/p>\n","protected":false},"excerpt":{"rendered":"In my previous post, I discussed several cases in which artificial intelligence (AI) chatbots encouraged people to take&hellip;\n","protected":false},"author":2,"featured_media":248180,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[59,57,58,50,56,54,55],"class_list":["post-248179","post","type-post","status-publish","format-standard","has-post-thumbnail","category-united-kingdom","tag-gb","tag-great-britain","tag-greatbritain","tag-news","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/248179","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=248179"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/248179\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/248180"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=248179"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=248179"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=248179"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}