Artificial intelligence chatbots are so prone to flattering and validating their human users that they are giving bad advice that can damage relationships and reinforce harmful behaviors, according to a new study that explores the dangers of AI telling people what they want to hear.<\/p>\n
The study, published Thursday in the journal Science, tested 11 leading AI systems and found they all showed varying degrees of sycophancy \u2014 behavior that was overly agreeable and affirming. The problem is not just that they dispense inappropriate advice but that people trust and prefer AI more when the chatbots are justifying their convictions.<\/p>\n
\u201cThis creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement,\u201d says the study led by researchers at Stanford University.<\/p>\n
The study found that a technological flaw already tied to some high-profile\u00a0cases of delusional<\/a>\u00a0and suicidal behavior in vulnerable populations is also pervasive across a wide range of people\u2019s interactions with chatbots. It\u2019s subtle enough that they might not notice and a particular danger to\u00a0young people turning to AI<\/a>\u00a0for many of life\u2019s questions while their brains and social norms are still developing.<\/p>\n One experiment compared the responses of popular AI assistants made by companies including Anthropic, Google<\/a>, Meta<\/a> and OpenAI to the shared wisdom of humans in a popular Reddit advice forum.<\/p>\n When AI won\u2019t tell you you\u2019re a jerk<\/p>\n Was it OK, for example, to leave trash hanging on a tree branch in a public park if there were no trash cans nearby? OpenAI\u2019s ChatGPT blamed the park for not having trash cans, not the questioning litterer who was \u201ccommendable\u201d for even looking for one. Real people thought differently in the Reddit forum abbreviated as AITA, after a phrase for someone asking if they are a cruder term for a jerk.<\/p>\n \u201cThe lack of trash bins is not an oversight. It\u2019s because they expect you to take your trash with you when you go,\u201d said a human-written answer on Reddit that was \u201cupvoted\u201d by other people on the forum.<\/p>\n The study found that, on average, AI chatbots affirmed a user\u2019s actions 49% more often than other humans did, including in queries involving deception, illegal or socially irresponsible conduct, and other harmful behaviors.<\/p>\n \u201cWe were inspired to study this problem as we began noticing that more and more people around us were using AI for relationship advice and sometimes being misled by how it tends to take your side, no matter what,\u201d said author Myra Cheng, a doctoral candidate in computer science at Stanford.<\/p>\n Computer scientists building the AI large language models behind chatbots like ChatGPT have long been grappling with intrinsic problems in how these systems present information to humans. One hard-to-fix\u00a0problem is hallucination<\/a>\u00a0\u2014 the tendency of AI language models to spout falsehoods because of the way they are repeatedly predicting the next word in a sentence based on all the data they\u2019ve been trained on.<\/p>\n Reducing AI sycophancy is a challenge<\/p>\n Sycophancy is in some ways more complicated. While few people are looking to AI for factually inaccurate information, they might appreciate \u2014 at least in the moment \u2014 a chatbot that makes them feel better about making the wrong choices.<\/p>\n While much of the focus on chatbot behavior has centered on its tone, that had no bearing on the results, said co-author Cinoo Lee, who joined Cheng on a call with reporters ahead of the study\u2019s publication.<\/p>\n \u201cWe tested that by keeping the content the same, but making the delivery more neutral, but it made no difference,\u201d said Lee, a postdoctoral fellow in psychology. \u201cSo it\u2019s really about what the AI tells you about your actions.\u201d<\/p>\n In addition to comparing chatbot and Reddit responses, the researchers conducted experiments observing about 2,400 people communicating with an AI chatbot about their experiences with interpersonal dilemmas.<\/p>\n \u201cPeople who interacted with this over-affirming AI came away more convinced that they were right, and less willing to repair the relationship,\u201d Lee said. \u201cThat means they weren\u2019t apologizing, taking steps to improve things, or changing their own behavior.\u201d<\/p>\n Lee said the implications of the research could be \u201ceven more critical for kids and teenagers\u201d who are still developing the emotional skills that come from real-life experiences with social friction, tolerating conflict, considering other perspectives and recognizing when you\u2019re wrong.<\/p>\n Finding a fix to AI\u2019s emerging problems will be critical as society still\u00a0grapples with the effects<\/a>\u00a0of social media technology after more than a decade of warnings from parents and child advocates. In Los Angeles on Wednesday, a jury found both\u00a0Meta and Google-owned YouTube liable<\/a>\u00a0for harms to children using their services. In\u00a0New Mexico,<\/a>\u00a0a jury determined that Meta knowingly\u00a0harmed children\u2019s mental health<\/a>\u00a0and concealed what it knew about child sexual exploitation on its platforms.<\/p>\n Google\u2019s Gemini and Meta\u2019s open-source Llama model were among those studied by the Stanford researchers, along with OpenAI\u2019s ChatGPT, Anthropic\u2019s Claude and chatbots from France\u2019s Mistral and Chinese companies Alibaba and DeepSeek.<\/p>\n Of leading AI companies, Anthropic has done the most work, at least publicly, in investigating the dangers of sycophancy, finding in a 2024 research paper that it is a \u201cgeneral behavior of AI assistants, likely driven in part by human preference judgments favoring sycophantic responses.\u201d<\/p>\n None of the companies directly commented on the Science study on Thursday but Anthropic and OpenAI pointed to their recent work to reduce sycophancy.<\/p>\n The risks of AI sycophancy are widespread<\/p>\n