{"id":601458,"date":"2026-05-24T13:27:22","date_gmt":"2026-05-24T13:27:22","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/601458\/"},"modified":"2026-05-24T13:27:22","modified_gmt":"2026-05-24T13:27:22","slug":"why-lawyers-keep-citing-fake-cases-invented-by-ai","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/601458\/","title":{"rendered":"Why lawyers keep citing fake cases invented by AI"},"content":{"rendered":"

In April the Alabama Supreme Court<\/a> sanctioned an attorney who had filed legal briefs laden with inaccurate citations generated by AI, including numerous references to cases that did not exist. After being informed he had cited a made-up precedent in one filing, the lawyer promised it wouldn\u2019t happen again\u2014but then cited \u201cnonexistent cases at the end of the very next sentence,\u201d as a justice noted in a concurring opinion. At least one other lawyer was sanctioned<\/a> that week for continuing to file AI-hallucinated material after being warned not to do so.<\/p>\n

A database maintained by Damien Charlotin<\/a>, a senior research fellow at the Paris School of Advanced Business Studies (HEC Paris), lists more than 1,400 cases where courts have addressed AI errors in the past three years, including filings by attorneys and self-represented litigants. As recently as last fall, Charlotin says, the list appeared to be growing exponentially. It\u2019s since leveled off to a steady flow of exasperated judicial rulings. \u201cFor the past two or three months, we have reached a plateau of around 350, 400 decisions a quarter,\u201d says Charlotin, who has also created an AI-powered reference checker called Pelaikan<\/a>.<\/p>\n

Courtroom proceedings are public, and lawyers face sanctions for false claims, making such errors comparatively easy to track. But uncaught errors in AI-generated material have also ensnared journalists<\/a>, software developers<\/a>, academic researchers<\/a> and government consultants<\/a>, some of whom have been well aware of AI\u2019s fallibility. On May 19 the New York Times reported that the author of The Future of Truth<\/a>, a book about how AI is shaping discourse, acknowledged his text contained more than a half-dozen fabricated or misattributed quotes produced by the technology.<\/p>\n

On supporting science journalism<\/p>\n

If you’re enjoying this article, consider supporting our award-winning journalism by subscribing<\/a>. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n

The pattern emerging across these cases is that people keep trusting AI\u2019s answers even when they know the systems can be wrong. So far, that misplaced trust has led to dismissed legal appeals, attorney fines, fired journalists and software outages<\/a>. Experts warn the stakes will rise as AI becomes more deeply embedded in professional work.<\/p>\n

\u201cHumans essentially have a tendency to believe that machines have more knowledge than they do, don\u2019t break and are infallible,\u201d says Alan Wagner, an associate professor of aerospace engineering at Pennsylvania State University.<\/p>\n

AI also appears to inspire a particular kind of trust. It can generate answers that are realistic-sounding but false in a way humans seldom do\u2014and people, it turns out, can find its guidance unusually believable. A study published this past February<\/a> asked participants to complete an image classification task with guidance they were told came from either humans or AI. The guidance\u2014no matter where it came from\u2014was right only half the time, but among participants who were told the advice came from AI, those with positive attitudes toward the technology performed worse than those who held less favorable views. No such effect appeared when participants were told the advice came from humans.<\/p>\n

\u201cThe results suggested that AI guidance has a quite specific ability to engender biases,\u201d says study co-author Sophie Nightingale, a senior lecturer in psychology at Lancaster University in England.<\/p>\n

Research co-authored by Wagner suggests<\/a> the problem could extend well beyond office work into life-or-death scenarios. In experiments inspired by drone warfare, his team asked participants to categorize images as civilians or enemy combatants and to choose whether to fire a missile at each potential target. A robot then provided feedback on each classification\u2014feedback that was, in fact, random\u2014and though participants\u2019 initial assessments were mostly accurate, they reversed their views in most cases where the bot disagreed. The scenario was a simulation, but participants were \u201cshown imagery of innocent civilians (including children), a UAV [uncrewed aerial vehicle] firing a missile, and devastation wreaked by a drone strike,\u201d according to the paper. They seemed to take the task seriously, says study co-author Colin Holbrook.<\/p>\n

\u201cI think that\u2019s the context in which those findings have to be interpreted,\u201d says Holbrook, an associate professor of cognitive and information sciences at the University of California, Merced. \u201cThese people were really trying. These people thought that it mattered,\u201d he adds. And if the scenario had been real, \u201cthey would have killed a lot of innocent people.\u201d<\/p>\n

Compared with earlier automation tools, today\u2019s AI handles a wider variety of tasks, such as generating computer programs and drafting legal briefs. That means more material to check, but it also means users can defer the thinking entirely to AI\u2014what researchers at the University of Pennsylvania\u2019s Wharton School recently called \u201ccognitive surrender<\/a>.\u201d In one of the team\u2019s experiments, participants received item-by-item feedback on a series of tasks and cash rewards for correct answers. Both practices reduced deference to faulty AI, but neither eliminated it, says Steven D. Shaw, a postdoctoral researcher at Wharton, who ran the study with associate professor of marketing Gideon Nave, also at Wharton.<\/p>\n

Educating AI users about the technology\u2019s limitations is another obvious approach, but efforts have produced limited results. As more than one judge has pointed out<\/a>, attorneys should by now know not to file AI-generated legal material without checking it, yet hallucinations keep showing up in court filings.<\/p>\n

Lab research has shown similarly modest effects<\/a> from warning messages. In one recent study<\/a>, researchers at Boston University \u201cinoculated\u201d students by alerting them that the AI chatbot ChatGPT tends to produce inaccurate summaries of academic sources and struggles with complex math and then asked them to complete related tasks using the tool. Participants warned about the source summaries were significantly more likely to verify the AI\u2019s output on that task. The warning had no significant effect on the math problems, where verification rates remained low. Some participants told the researchers they came in trusting AI\u2019s mathematical abilities; some said the experiment\u2019s time constraints, which were built in to mimic real-world deadlines, cut into how often they verified results.<\/p>\n

\u201cOur findings suggest that awareness alone isn\u2019t enough,\u201d writes study co-author Chi B. Vu, a graduate student in human-AI interaction at BU\u2019s Division of Emerging Media Studies, in an e-mail to Scientific American. \u201cThe message wasn\u2019t ignored exactly; it was overridden by competing pressures and trust in certain tasks conducted by [generative] AI.\u201d<\/p>\n

Warnings about AI accuracy also compete with advertising that highlights the technology\u2019s potential and with workplace pressures to use it to save time. And as AI improves at many tasks, users may grow less inclined to double-check it at all. That can keep them from seeing the errors that remain, further deepening their confidence.<\/p>\n

\u201cThey don\u2019t ever get to the ground truth,\u201d Nightingale says. \u201cThey don\u2019t have any reason to question it because they carry on in their lives thinking that AI tool is correct\u2014because \u2018Why wouldn\u2019t it be?\u2019\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"In April the Alabama Supreme Court sanctioned an attorney who had filed legal briefs laden with inaccurate citations…\n","protected":false},"author":2,"featured_media":601459,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":["post-601458","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-technology","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/601458","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=601458"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/601458\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/601459"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=601458"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=601458"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=601458"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}