{"id":215923,"date":"2026-01-02T03:15:10","date_gmt":"2026-01-02T03:15:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/il\/215923\/"},"modified":"2026-01-02T03:15:10","modified_gmt":"2026-01-02T03:15:10","slug":"the-core-problem-with-large-language-models","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/il\/215923\/","title":{"rendered":"The Core Problem with Large Language Models"},"content":{"rendered":"

I have shared many examples of the unreliability of OpenAI\u2019s ChatGPT, Google\u2019s Gemini, and other large language models (LLMs). Recent examples include<\/a> evaluating a rotated tic-tac-toe grid, and labeling a possum\u2019s body parts. Some correspondents have objected that no one is going to ask an LLM to evaluate an obviously pointless tic-tac-toe rotation or to identify a possum\u2019s tail and snout.<\/p>\n

I should have been clearer. The many examples I\u2019ve reported are not just \u201cgotchas\u201d meant to embarrass LLMs. They are intended to demonstrate in a clear and compelling way that LLMs are inherently unreliable.<\/p>\n

Going back to my Oxford University Press tetralogy, The AI Delusion<\/a>, The 9 Pitfalls of Data Science<\/a> (co-authored with Jay Cordes), The Phantom Pattern Problem<\/a>, and Distrust:<\/a> Big Data, Data-Torturing, and the Assault on Science,<\/a> I have argued relentlessly that AI systems based on data mining are unreliable.<\/p>\n

Why the LLM system is inherently unreliable<\/p>\n

The scientific method begins with a specific hypothesis that is tested with data, ideally a randomized controlled trial. Data mining goes in the other direction, beginning with data and looking to find what have been called<\/a> \u201chidden patterns and secret correlations.\u201d When a heretofore unknown pattern or correlation is found, data miners either<\/a> concoct a plausible theory or argue that theories are not needed. For example, in a Wired piece titled, \u201cThe End of Theory: The Data Deluge Makes the Scientific Method Obsolete<\/a>,\u201d Chris Anderson wrote,<\/p>\n

Correlation supersedes causation, and science can advance even without coherent models, unified theories, or really any mechanistic explanation at all.<\/p>\n

The problem is that the number of genuinely useful relationships is limited. But the data deluge is increasing the number of coincidental, useless statistical patterns exponentially \u2014 so the probability that a data-mined pattern is actually useful is getting ever closer to zero. As I have written elsewhere<\/a>:<\/p>\n

This is the paradox of big data: It would seem that having data for a large number of variables will help us find more reliable patterns; however, the more variables we consider, the less likely it is that what we find will be useful.<\/p>\n

Data-mining algorithms look for statistical patterns with no idea how the data relate to the real world. The data might as well be labeled &4qp#L, M&2V+v, or any other randomly generated collection of ASCII characters. Data-mining algorithms are consequently unable to distinguish between useful causation and useless correlation. Here are a few examples of misplaced faith in data mining:<\/p>\n

\"\"Amazon book cover for The AI Delusion by Gary Smith<\/p>\n

Admiral Insurance, Britain\u2019s largest car insurance company, planned to launch<\/a> firstcarquote, which would base its car insurance rates on a data-mined analysis of an applicant\u2019s Facebook posts \u2014 before being blocked by Facebook lawyers.<\/p>\n

Yongqianbao, a Chinese tech company, reported that<\/a> they had developed a data-mining algorithm that evaluates loan applications based on applicants\u2019 smartphone usage, including such measures as whether they keep their phones fully charged and how many calls go unanswered. The company is no longer in business.<\/p>\n

Gild developed data-mining software for evaluating applicants<\/a> for software engineering jobs by monitoring their online activities. Even the chief scientist acknowledged that some of the factors chosen by its data-mining software do not make sense, saying, \u201cObviously, it\u2019s not a causal relationship.\u201d A former employee wrote that, \u201cCustomers really, really hate the product. There are almost no customers that have an overall positive experience.\u201d<\/p>\n

A JP Morgan study<\/a> concluded that Trump tweets containing the words China, billion, products, Democrats, or great have a \u201cstatistically significant impact\u201d on interest rates. They don\u2019t.<\/p>\n

Researchers found<\/a> a correlation between a three-week moving average of Google searches for the word debt and changes in the Dow Jones Industrial Average. Using data for a 7-year period, they reported that their debt strategy had an astounding 23.0% annual return, compared with 2.2% for a buy-and-hold strategy. However, for the 7 years after their study, their debt strategy had an annual return of 2.81%, compared with 8.60% for buy-and-hold.<\/p>\n

The perils of data mining<\/p>\n

LLMs mine data but they search for statistical patterns in words instead of numbers. Similar to traditional data-mining algorithms, LLMs do not know how the text they input and output relate to the real world and consequently have no way of knowing whether the statements they process and generate are true or false, sensible or nonsense.<\/p>\n

Here are a few examples of mishaps caused by this core problem:<\/p>\n

Air Canada was sued and lost a case<\/a> in which a customer-service chatbot generated incorrect information about the airline\u2019s bereavement-fare policy.<\/p>\n

A car dealer\u2019s ChatGPT bot agreed to sell<\/a> a $76,000 Chevy Tahoe for $1. The bot said, \u201cThat\u2019s a deal, and that\u2019s a legally binding offer \u2013 no takesies backsies,\u201d but the dealer refused to honor the deal.<\/p>\n

A New York Federal court sanctioned<\/a> experienced lawyers who used ChatGPT to generate a court filing that cited non-existence cases.<\/p>\n

A Greek woman filed for divorce<\/a> because a ChatGPT reading of photos of coffee grounds left in her cup and her husband\u2019s cup indicated that he was having an affair with a \u201chome wrecker.\u201d<\/p>\n

LLM-based hiring systems have been shown<\/a> to generate outcomes that are biased with respect to race and sex.<\/p>\n

A study of the use of OpenAI\u2019s Whisper transcription tool<\/a> to create records of doctor-patient interactions found that it often created entirely made-up phrases and sentences, many of which were potentially harmful.<\/p>\n

A man was reportedly<\/a> hospitalized for severe psychiatric problems after substituting sodium bromide for table salt following a ChatGPT session.<\/p>\n

The parents of a 16-year-old who hung himself filed a wrongful-death lawsuit<\/a> against OpenAI, alleging that ChatGPT had given their son a \u201cstep-by-step playbook\u201d on how to hang himself, including advice on the best knot to use and offering to write a suicide note for him.<\/p>\n

Scaling up (by increasing the number of parameters, the size of the training data, and the computational power) won\u2019t solve this fundamental weakness \u2014 that LLMs are unreliable because they do not know how words relate to the real world.<\/p>\n

Why there is no simple fix<\/p>\n

Expert training can help LLMs give better answers to the prompts they are trained on but (1) the experts cannot anticipate all prompts an LLM might be asked, (2) many (most?) real world decisions involve subjective probabilities that cannot be reliably generated by clueless LLMs and cannot be anticipated by human trainers, and (3) following instructions is not general intelligence.<\/p>\n

The point of the hundreds of examples I have given over the years is that predictions, advice, or conclusions based solely on statistical patterns are unreliable. Data mining is dodgy because correlation does not supersede causation.<\/p>\n","protected":false},"excerpt":{"rendered":"I have shared many examples of the unreliability of OpenAI\u2019s ChatGPT, Google\u2019s Gemini, and other large language models…\n","protected":false},"author":2,"featured_media":215924,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[345,343,344,85,46,125],"class_list":{"0":"post-215923","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-il","12":"tag-israel","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/215923","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=215923"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/215923\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/215924"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=215923"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=215923"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=215923"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}