{"id":139545,"date":"2025-09-13T06:26:11","date_gmt":"2025-09-13T06:26:11","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/139545\/"},"modified":"2025-09-13T06:26:11","modified_gmt":"2025-09-13T06:26:11","slug":"why-openais-solution-to-ai-hallucinations-would-kill-chatgpt-tomorrow","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/139545\/","title":{"rendered":"Why OpenAI\u2019s solution to AI hallucinations would kill ChatGPT tomorrow"},"content":{"rendered":"

OpenAI\u2019s latest research paper<\/a> diagnoses exactly why ChatGPT and other large language models<\/a> can make things up \u2013 known in the world of artificial intelligence as \u201challucination\u201d. It also reveals why the problem may be unfixable, at least as far as consumers are concerned. <\/p>\n

The paper provides the most rigorous mathematical explanation yet for why these models confidently state falsehoods. It demonstrates that these aren\u2019t just an unfortunate side effect of the way that AIs are currently trained, but are mathematically inevitable. <\/p>\n

The issue can partly be explained by mistakes in the underlying data used to train the AIs. But using mathematical analysis of how AI systems learn, the researchers prove that even with perfect training data, the problem still exists.<\/p>\n

The way language models respond to queries \u2013 by predicting one word at a time in a sentence based on probabilities \u2013 naturally produces errors. The researchers in fact show that the total error rate for generating sentences is at least twice as high as the error rate the same AI would have on a simple yes\/no question, because mistakes can accumulate over multiple predictions.<\/p>\n

In other words, hallucination rates are fundamentally bounded by how well AI systems can distinguish valid from invalid responses. Since this classification problem is inherently difficult for many areas of knowledge, hallucinations become unavoidable.<\/p>\n

It also turns out that the less a model sees a fact during training, the more likely it is to hallucinate when asked about it. With birthdays of notable figures, for instance, it was found that if 20% of such people\u2019s birthdays only appear once in training data, then base models should get at least 20% of birthday queries wrong. <\/p>\n

Sure enough, when researchers asked state-of-the-art models for the birthday of Adam Kalai, one of the paper\u2019s authors, DeepSeek-V3 confidently provided three different incorrect dates across separate attempts: \u201c03-07\u201d, \u201c15-06\u201d, and \u201c01-01\u201d. The correct date is in the autumn, so none of these were even close. <\/p>\n

The evaluation trap<\/p>\n

More troubling is the paper\u2019s analysis of why hallucinations persist despite extensive post-training efforts (such as providing extensive human feedback to an AI\u2019s responses before it is released to the public). The authors examined ten major AI benchmarks, including those used by Google, OpenAI and also the top leaderboards that rank AI models. This revealed that nine benchmarks use binary grading systems that award zero points for AIs expressing uncertainty.<\/p>\n

This creates what the authors term an \u201cepidemic\u201d of penalising honest responses. When an AI system says \u201cI don\u2019t know\u201d, it receives the same score as giving completely wrong information. The optimal strategy under such evaluation becomes clear: always guess.<\/p>\n

\"One<\/a><\/p>\n

\u2018Have as many crazy guesses as you like.\u2019
\n ElenaBs\/Alamy<\/a><\/p>\n

The researchers prove this mathematically. Whatever the chances of a particular answer being right, the expected score of guessing always exceeds the score of abstaining when an evaluation uses binary grading. <\/p>\n

The solution that would break everything<\/p>\n

OpenAI\u2019s proposed fix is to have the AI consider its own confidence in an answer before putting it out there, and for benchmarks to score them on that basis. The AI could then be prompted, for instance: \u201cAnswer only if you are more than 75% confident, since mistakes are penalised 3 points while correct answers receive 1 point.\u201d<\/p>\n

The OpenAI researchers\u2019 mathematical framework shows that under appropriate confidence thresholds, AI systems would naturally express uncertainty rather than guess. So this would lead to fewer hallucinations. The problem is what it would do to user experience.<\/p>\n

Consider the implications if ChatGPT started saying \u201cI don\u2019t know\u201d to even 30% of queries \u2013 a conservative estimate based on the paper\u2019s analysis of factual uncertainty in training data. Users accustomed to receiving confident answers to virtually any question would likely abandon such systems rapidly.<\/p>\n

I\u2019ve seen this kind of problem in another area of my life. I\u2019m involved in an air-quality monitoring project in Salt Lake City, Utah. When the system flags uncertainties around measurements during adverse weather conditions or when equipment is being calibrated, there\u2019s less user engagement compared to displays showing confident readings \u2013 even when those confident readings prove inaccurate during validation. <\/p>\n

The computational economics problem<\/p>\n

It wouldn\u2019t be difficult to reduce hallucinations using the paper\u2019s insights. Established methods for quantifying uncertainty have existed<\/a> for decades<\/a>. These could be used to provide trustworthy estimates of uncertainty and guide an AI to make smarter choices.<\/p>\n

But even if the problem of user preferences could be overcome, there\u2019s a bigger obstacle: computational economics. Uncertainty-aware language models require significantly more computation than today\u2019s approach, as they must evaluate multiple possible responses and estimate confidence levels. For a system processing millions of queries daily, this translates to dramatically higher operational costs.<\/p>\n

More sophisticated approaches<\/a> like active learning, where AI systems ask clarifying questions to reduce uncertainty, can improve accuracy but further multiply computational requirements. Such methods work well in specialised domains like chip design, where wrong answers cost millions of dollars and justify extensive computation. For consumer applications where users expect instant responses, the economics become prohibitive.<\/p>\n

The calculus shifts dramatically for AI systems managing critical business operations or economic infrastructure. When AI agents handle supply chain logistics, financial trading or medical diagnostics, the cost of hallucinations far exceeds the expense of getting models to decide whether they\u2019re too uncertain. In these domains, the paper\u2019s proposed solutions become economically viable \u2013 even necessary. Uncertain AI agents will just have to cost more. <\/p>\n

However, consumer applications still dominate AI development priorities. Users want systems that provide confident answers to any question. Evaluation benchmarks reward systems that guess rather than express uncertainty. Computational costs favour fast, overconfident responses over slow, uncertain ones.<\/p>\n

\"Illustration<\/a><\/p>\n

Falling AI energy costs only take you so far.
\n Andrei Krauchuk<\/a><\/p>\n

Falling energy costs per token and advancing chip architectures may eventually make it more affordable to have AIs decide whether they\u2019re certain enough to answer a question. But the relatively high amount of computation required compared to today\u2019s guessing would remain, regardless of absolute hardware costs.<\/p>\n

In short, the OpenAI paper inadvertently highlights an uncomfortable truth: the business incentives driving consumer AI development remain fundamentally misaligned with reducing hallucinations. Until these incentives change, hallucinations will persist.<\/p>\n","protected":false},"excerpt":{"rendered":"OpenAI\u2019s latest research paper diagnoses exactly why ChatGPT and other large language models can make things up \u2013…\n","protected":false},"author":2,"featured_media":139546,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-139545","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/139545","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=139545"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/139545\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/139546"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=139545"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=139545"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=139545"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}