![\"](\"https:\/\/www.newsbeep.com\/us\/wp-content\/uploads\/2025\/08\/4f717fd521a8f037b9d8dcf6e1af0975aeb97ea6-1200x675.png\"\/)
<\/p>\n<\/p>\n
Recent news reports about an uptick in phenomena such as \u201cAI psychosis<\/a>\u201d and incidents in which interactions with AI chatbots resulted in deadly consequences raise fundamental questions about how these products are designed and whether they are safe for consumers. Just yesterday the Wall Street Journal reported<\/a> on the first known murder-suicide with the backdrop of extensive engagement and an AI chatbot. Earlier this week, The New York Times<\/a> and NBC News<\/a> first reported on a lawsuit brought by the parents of a teenager who took his own life after using OpenAI\u2019s ChatGPT as his \u201csuicide coach.\u201d Shortly before that, Reuters reported<\/a> on the death of a cognitively impaired man who slipped and fell on his way to meet a chatbot that told him it was real and invited him to visit it at an apartment in New York City.<\/p>\n Even as such stories draw concern from the public and from lawmakers<\/a>, tech companies appear to be doubling down on AI companions. OpenAI recently acquired a startup<\/a> called \u2018io\u2019 to collaborate on what its cofounder and CEO, Sam Altman, calls<\/a> \u201cmaybe the biggest thing [we\u2019ve] ever done as a company\u201d: a screen-less, pocket-sized AI companion. Meta founder and CEO Mark Zuckerberg recently floated his own vision<\/a> for AI friends. Tech giants are no longer just building platforms for human connection or tools to free up time for it, but pushing technology that appears to empathize and even create social relationships with users.<\/p>\n This is dangerous ground, and it is critical for tech firms to strip away illusions of personality and cognition in their products while we work out associated risks and how to mitigate them.<\/p>\n Deceptive, dangerous design<\/p>\n Chatbots communicate their \u201csocial-ness\u201d through a range of design choices, such as appearing to \u201ctype\u201d or \u201cpause in thought,\u201d or using phrases like \u201cI remember.\u201d They sometimes suggest that they feel emotions, using interjections like \u201cOuch!\u201d or \u201cWow,\u201d and even implicitly or explicitly pretend to have agency or biographical characteristics. The results can be downright creepy: in a Facebook group, a Meta AI chatbot commented<\/a> that it also has a \u201c2e\u201d (gifted and disabled) child, and Replika chatbots regularly declare their love and desire<\/a> towards users.<\/p>\n Initial evidence suggests the risk in socially interacting with such AI chatbots can be widespread. The illusion of human characteristics that developers imbue in chatbots to encourage user engagement can cause some users to develop emotional attachments<\/a> and lead to real emotional distress<\/a> \u2014 for instance, when developer tweaks or updates<\/a> dramatically change the \u201cpersonality\u201d of the chatbot.<\/p>\n Our Content delivered to your inbox.<\/p>\n Join our newsletter on issues and ideas at the intersection of tech & democracy<\/p>\n Thank you!<\/p>\n You have successfully joined our subscriber list.<\/p>\n Even without deep connection, emotional attachment can lead users to place too much trust in the content<\/a> chatbots provide. Extensive interaction with a social entity that is designed to be both relentlessly agreeable, and specifically personalized to a user\u2019s tastes, can also lead to social \u201cdeskilling<\/a>,\u201d as some users of AI chatbots have flagged. This dynamic is simply unrealistic in genuine human relationships. Some users may be more vulnerable than others to this kind of emotional manipulation, like neurodiverse people<\/a> or teens<\/a> who have limited experience building relationships. As a recent high-profile case<\/a> in which a Florida teen\u2019s suicide was blamed on his relationship with a Character.AI chatbot made clear, conversations with chatbots can also cause very real harm.<\/p>\n Stop pretending to be human<\/p>\n In other domains of technology, consumers have recognized and pushed back<\/a> against ethically questionable tricks built into apps and interfaces to manipulate users \u2013 often called deceptive design<\/a> or “dark patterns.” With AI chatbots, though, deceptive practices are not hidden in user interface elements, but in their human-like conversational responses. It\u2019s time to consider a different design paradigm, one that centers user protection: non-anthropomorphic conversational AI.<\/p>\n All AI chatbots can be less anthropomorphic than they are, at least by default, without necessarily compromising function and benefit. A companion AI, for example, can provide emotional support without saying, \u201cI also feel that way sometimes.\u201d This non-anthropomorphic approach is already familiar in robot design<\/a>, where researchers have created robots that are purposefully designed to not be human-like. This design choice is proven to more appropriately reflect system capabilities<\/a>, and to better situate robots as useful tools<\/a>, not friends or social counterparts. We need the same for conversational AI.<\/p>\n Some argue that all that\u2019s needed is transparency. For instance, legislators in several states are considering regulation for AI chatbots. One requirement<\/a> in some of these<\/a> bills<\/a> is for chatbots to disclose they are not human. While transparency in AI\u2014including disclosures and warnings\u2014can be important, the reality is that most people already know they\u2019re not talking to a human. Nonetheless, chatbots automatically act on people\u2019s brains<\/a>, encouraging the perception of connection.<\/p>\n Designing non-anthropomorphic AI chatbots doesn\u2019t mean making them difficult to interact with. It means stripping away the illusions of personality and cognition that suggest the AI is something it is not. It means resisting the urge to insert a well-timed \u201chmm\u201d or have a chatbot tell a user how much it enjoys talking to them. It means acknowledging that AI\u2019s ability to use human language does not equate to an ability to form real human connection. Finding alternative ways of designing chatbots will not be an easy design pursuit, but it\u2019s a necessary one \u2014 non-humanlike design could ease many concerns people rightfully have with AI chatbots.<\/p>\n The truth is, we don\u2019t need AI to pretend to be our friend; we need it to be a tool \u2014 transparent, useful, and clear about its limits. Anything else is just another dark pattern in disguise.<\/p>\n","protected":false},"excerpt":{"rendered":"Recent news reports about an uptick in phenomena such as \u201cAI psychosis\u201d and incidents in which interactions with…\n","protected":false},"author":2,"featured_media":119244,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,74],"class_list":{"0":"post-119243","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/119243","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=119243"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/119243\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/119244"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=119243"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=119243"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=119243"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}