{"id":486730,"date":"2026-10-06T09:04:08","date_gmt":"2026-10-06T09:04:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-ca\/486730\/"},"modified":"2026-10-06T09:04:08","modified_gmt":"2026-10-06T09:04:08","slug":"ai-chatbots-are-overconfident-like-humans-haas-news","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-ca\/486730\/","title":{"rendered":"AI chatbots are overconfident\u2014like humans – Haas News"},"content":{"rendered":"

\"\"Image: Marcela Barsse<\/a> for iStock<\/p>\n

New UC Berkeley Haas research testing 11 leading large language models\u2014including models from OpenAI, Google, Anthropic, and Meta\u2014found they were overconfident on average: the models reported 88% confidence but were correct only 79% of the time, a nine-point overconfidence gap.
\nLike humans, the models grew more overconfident as questions got harder and underconfident on easier ones, a pattern well documented in humans and known as the \u201chard-easy effect.\u201d
\nThe research suggests human-feedback training (RLHF) may reward more confident answers over those that are hedged; newer reasoning models were better calibrated but still imperfect.<\/p>\n

It is no secret that AI chatbots hallucinate. Lawyers have filed AI-generated court briefs citing made-up cases. Students have turned in papers footnoted with studies never published. Reporters and researchers alike have seen chatbots confidently name sources and statistics that simply don\u2019t exist.<\/p>\n

Don Moore, a UC Berkeley Haas management professor who has spent his career studying human overconfidence, went into his research on AI assuming this kind of confident fabrication was a \u201cperversity\u201d unique to machines\u2014the behavior of a system that hadn\u2019t yet learned to reason properly. But the deeper he delved, the less it looked like a machine problem at all. AI chatbots, he came to realize, are trained to be compliant, agreeable, and quick to produce an answer rather than admit they don\u2019t have one. Humans, when pushed to appear competent and cooperative, do the same thing.<\/p>\n

\u201cIt\u2019s not that different from when humans are confident and wrong,\u201d says Moore, the Lorraine Tyson Mitchell Chair in Leadership and Communication. \u201cWe do it routinely.\u201d<\/p>\n

It\u2019s not that different from when humans are confident and wrong. We do it routinely.\u201d<\/p>\n

\u2014Professor Don Moore<\/p>\n

That realization sits at the heart of his new working paper<\/a>, currently under review at the journal Cognitive Science. Coauthored by professor Jacob Bien of the USC Marshall School of Business and recent UC Berkeley data science graduates Noam Michael, BA 26, and Daniel BenShushan, BA 26, the study tests 11 leading Large Language Models (LLMs)\u2014including models from OpenAI, Google, Anthropic, and Meta\u2014on their ability to know what they don\u2019t know. The research finds that the LLMs they studied are, on average, overconfident. And like people, the AI models grow more overconfident as questions get harder and oddly under-confident when questions are easier.<\/p>\n

The findings raise uncomfortable questions about trust. The paper notes many users already rely on chatbots heavily and unquestioningly, trust that may be misplaced if a model\u2019s confidence outpaces its accuracy.\u00a0Moore fears this same blind spot could show up in self-driving cars that don\u2019t know when to hesitate, or in autonomous weapons systems already in development, where an AI\u2019s failure to register its own uncertainty isn\u2019t just misleading but dangerous.<\/p>\n

Calibration versus accuracy<\/p>\n

Most people judge a chatbot\u2019s answers in binary terms: they are right, or hallucinations. Moore\u2019s research measures not whether AI gets things right but whether it\u00a0knows\u00a0when it\u2019s likely wrong. Psychologists call this calibration\u2014holding a belief with a level of confidence that actually matches how likely it is to be true. A well-calibrated mind, human or machine, sounds sure about the sun rising tomorrow but will hedge on a coin-flip election.<\/p>\n

This is different from accuracy. A model could get 90% of answers right and still be badly calibrated if it claimed near certainty on every one, including the 10% it gets wrong. Poor calibration leaves users unable to tell which answers deserve a second look. Part of the problem is rooted in how large language models work: they generate text by predicting likely next words rather than looking up verified facts. Every response, confident-sounding or not, is ultimately an educated guess.<\/p>\n

To measure AI calibration with precision, Moore\u2019s team devised a new test they call LifeEval. It gives an AI a person\u2019s sex and age, asks it to guess how long that person will live, and checks its stated confidence against government life-expectancy data. By tweaking the person\u2019s age or how close the guess needs to be, researchers could make the test harder or easier without changing anything else about it\u2014solving a problem with older tests in which hard and easy questions differed in too many ways at once. LifeEval lets researchers isolate one clean question: as difficulty changes, does AI\u2019s confidence rise and fall the way it should?<\/p>\n

Across all 11 models tested, the pattern was strikingly consistent. On average, models reported 88% confidence in their answers but were correct only 79% of the time, a nine-point overconfidence gap. The researchers suggest AI exhibits the \u201chard-easy effect\u201d for the same reason people do: confidence is a noisy, imperfect signal of real performance, so it drifts toward a default middle ground, leaving models overconfident on hard questions and too modest on easy ones.\u00a0<\/p>\n

Newer \u201creasoning\u201d models, built to deliberate and self-critique before answering, were meaningfully better calibrated than faster chat models, though still imperfect. Preliminary comparisons, Moore says, suggest AI\u2019s overconfidence is somewhat milder than typical human overconfidence, a finding he is continuing to study.<\/p>\n

The core issue, Moore\u2019s research suggests, isn\u2019t that models perform badly; it\u2019s that they don\u2019t know what they don\u2019t know, any more than people do. When a model wanders into unfamiliar territory, it tends to answer with the same assurance it uses on questions it handles well, leaving users unable to distinguish the two.<\/p>\n

\u201cWhat does overconfidence look like?\u201d asks Moore. \u201cIt looks like confidence.\u201d<\/p>\n

What does overconfidence look like? It looks like confidence.\u201d<\/p>\n

\u2014Professor Don Moore<\/p>\n

The training trade-off<\/p>\n

Part of the explanation, Moore argues, traces back to how these models are built. Before public release, AI systems go through \u201creinforcement learning from human feedback,\u201d or RLHF, a fine-tuning process in which human evaluators rate model responses, nudging the system toward answers people prefer. The trouble is that people tend to prefer confident, agreeable answers over hedged, uncertain ones. That preference may be training overconfidence directly into the system\u2014models emerging from RLHF may be more compliant and more assured than before training. It exposes a real tension between an AI built to be liked and one built to be accurate, and Moore suggests the likely resolution isn\u2019t a single fix, but different classes of AI for different purposes.<\/p>\n

\u201cMy guess is that there will be different sorts of AI systems, some designed for support and therapy and others designed for industrial applications, whose goal is to be useful,\u201d says Moore. \u201cUseful systems should avoid flattery and sycophancy whenever possible. They should speak the truth directly and as accurately as they know how.\u201d<\/p>\n

In the meantime, users can take small steps on their own. Moore says he routinely prompts his own AI assistant to accompany each answer with a calibrated probability and to flag its own uncertainty rather than present every answer as certain. \u201cIt\u2019s pretty good about noting\u2014here\u2019s where the evidence is good, here\u2019s what I\u2019m sure about\u2014but you might want to check this,\u201d says Moore.<\/p>\n

Users, he adds, should treat AI as they would any fallible source of information. \u201cTrust,\u201d he says, \u201cbut verify.\u201d<\/p>\n

Read the full working paper:<\/p>\n

Confidence Calibration in Large Language Models<\/a>
Noam Michael, Daniel BenShushan, Jacob Bien, & Don A. Moore<\/p>\n","protected":false},"excerpt":{"rendered":"Image: Marcela Barsse for iStock New UC Berkeley Haas research testing 11 leading large language models\u2014including models from…\n","protected":false},"author":2,"featured_media":486731,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[2080,1979,143,145,144,82000],"class_list":["post-486730","post","type-post","status-publish","format-standard","has-post-thumbnail","category-oakland","tag-artificial-intelligence","tag-latest-news","tag-oakland","tag-oakland-headlines","tag-oakland-news","tag-top-research-news"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/486730","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=486730"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/486730\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/486731"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=486730"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=486730"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=486730"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}