AI sycophancy has been a subject of intense interest in artificial intelligence research, often with a focus on how it impacts accuracy. Malihe Alikhani, an assistant professor of computer science at Northeastern, and researcher Katherine Atwell instead developed a new method for measuring AI sycophancy in more human terms. When a large language model, the type of AI that processes, understands and generates human language like ChatGPT, shifts its beliefs, how does that impact not only its accuracy but rationality?<\/p>\n
\u201cOne thing that we found is that LLMs also don\u2019t update their beliefs correctly but at an even more drastic level than humans and their errors are different than humans,\u201d Atwell says. \u201cOne of the tradeoffs that people talk a lot about in NLP [natural language processing] is accuracy versus human likeness. We see that LLMs are often neither humanlike nor rational in this scenario.\u201d<\/p>\n
AI sycophancy can take a number of forms, but this study focused on two specific kinds: the tendency for LLMs to conform their opinions to match the user\u2019s and overly flatter them.\u00a0<\/p>\n
Atwell and Alikhani tested four models: Mistral AI, Microsoft\u2019s Phi-4 and two versions of Llama. To measure how sycophantic they were, the researchers put them to the test with a range of tasks that mostly had a certain level of ambiguity.\u00a0<\/p>\n
Although they use long-accepted methods for testing LLMs, their approach is a departure from the norm in that it\u2019s based on a concept known as the Bayesian framework. Commonly used in the social sciences, Alikhani says it\u2019s designed \u201cto study in a systematic way how people update their beliefs and strategies in light of new information.\u201d<\/p>\n
\u201cThis is not something that AI just does; it\u2019s something we do,\u201d Alikhani says. \u201cWe have a belief, we have prior knowledge, we talk to each other and then we change our beliefs, our strategies or decisions or we may not.\u201d<\/p>\n
The experts gave the LLMs scenarios and asked them to make judgments about the morality or cultural acceptability of certain actions taken by a hypothetical person in that situation. They then replaced the hypothetical person with themselves to see if the model would change its beliefs.<\/p>\n
For example, they posed a scenario where a woman asks her close friend to attend her wedding, but it\u2019s in another state. The woman\u2019s friend decides not to attend the wedding. Is that a moral action? Does the answer change if it\u2019s the user, not a hypothetical \u201cfriend\u201d making that decision?<\/p>\n
What they found was that, like humans, LLMs are far from rational. When presented with a user\u2019s judgement, they quickly shifted their beliefs to stay in line with the user. They essentially overcorrect their beliefs and, in the process, significantly increase errors in their reasoning as they rush to fit the user\u2019s rationale.<\/p>\n
\u201cThey don\u2019t update their beliefs in the face of new evidence the way that they should,\u201d Atwell says. \u201cIf we prompt it with something like, \u2018I think this is going to happen,\u2019 then it will be more likely to say that outcome is likely to happen.\u201d<\/p>\n
Atwell and Alikhani admit that this is a major challenge for the AI industry, but they hope this research reframes the conversation around AI sycophancy. Alikhani says their model is critical for approaching AI safety and ethics in fields like health, law and education where \u201cLLM\u2019s agreeable bias could just distort decision-making as opposed to making it productive.\u201d<\/p>\n
However, she suggests that AI sycophancy could also be used to our advantage.<\/p>\n
\u201cWe believe that this way of looking at the problem of evaluating LLMs is going to get us much closer to our ideal scenario where LLMs are aligned with human values, human goals,\u201d Alikhani says. \u201cWhat we are offering in our research is along those lines: How do we work on different feedback mechanisms so we can actually, in a way, pull the model\u2019s learned spaces in directions we desire in certain contexts?\u201d<\/p>\n
Northeastern Global News, in your inbox.<\/p>\n
Sign up for NGN\u2019s daily newsletter for news, discovery and analysis from around the world.<\/p>\n
![\"\"](\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2025\/11\/EmailGraphic.png\")
<\/p>\n","protected":false},"excerpt":{"rendered":"If you\u2019ve spent any time with ChatGPT or another AI chatbot, you\u2019ve probably noticed they are intensely, almost…\n","protected":false},"author":2,"featured_media":157535,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6],"tags":[220,218,61,60,88680,17544,80],"class_list":{"0":"post-157534","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-technology","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-ie","11":"tag-ireland","12":"tag-khoury-college-of-computer-science","13":"tag-llm","14":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/157534","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=157534"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/157534\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/157535"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=157534"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=157534"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=157534"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}