{"id":322565,"date":"2026-05-19T08:40:33","date_gmt":"2026-05-19T08:40:33","guid":{"rendered":"https:\/\/www.newsbeep.com\/us-ca\/322565\/"},"modified":"2026-05-19T08:40:33","modified_gmt":"2026-05-19T08:40:33","slug":"california-lawyers-cant-quit-ai-even-as-hallucinations-pile-up","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us-ca\/322565\/","title":{"rendered":"California lawyers can\u2019t quit AI\u00a0\u2014 even as hallucinations pile up"},"content":{"rendered":"<p><img alt=\"Fake citations generated by AI, known as \u201challucinations,\u201d can rack up punishments and fines for lawyers.\" loading=\"eager\" fetchpriority=\"high\"   style=\"aspect-ratio:3 \/ 2\" class=\"x100 y100 opc bgpc ofcv bgscv block bg-gray200 mnh0px fill\"\/><\/p>\n<p>Fake citations generated by AI, known as \u201challucinations,\u201d can rack up punishments and fines for lawyers.<\/p>\n<p>John Blanchard\/S.F. Chronicle<\/p>\n<p>A lawsuit against Bay Area Rapid Transit by an officer who was fired after refusing to be vaccinated against COVID-19 faced possible dismissal this spring because the officer\u2019s lawyer had fallen ill and missed numerous deadlines. When the lawyer resurfaced, she filed arguments explaining her absence and why it shouldn\u2019t be used against her client\u00a0\u2014 but three of the four cases she cited as precedent, a federal magistrate found, were nonexistent.<\/p>\n<p>The fictitious cases, like many others uncovered by courts in recent years, were evidently the product of artificial intelligence, U.S. Magistrate Judge Thomas Hixson said <a href=\"https:\/\/websitedc.s3.amazonaws.com\/documents\/Gregoire_v._BART_USA_6_May_2026.pdf\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">in a ruling this month<\/a>. He fined attorney Jessica Barsotti $1,000 and ordered her to take an hour of training in \u201cthe ethical use of artificial intelligence in the practice of law.\u201d<\/p>\n<p>\u201cThe Court does not prohibit or oppose the use of artificial intelligence in legal advocacy, so long as counsel independently verifies the accuracy of AI-generated content,\u201d Hixson wrote.\u00a0<\/p>\n<p class=\"uiTextSmall f aic jcc\">Article continues below this ad<\/p>\n<p>He did not identify the nonexistent cases, saying he didn\u2019t want to imply that they were real. But one of Barsotti\u2019s filings last month, arguing that a lawyer\u2019s inadvertent errors or neglect shouldn\u2019t be used to punish her client, relied on three purported rulings by federal courts between 1964 and 2003 that could not be found in official records of published decisions.<\/p>\n<p>Barsotti, who missed nearly four months of law practice because of illness and other problems, could not be reached for comment. If her case filings were the product of AI misinformation, they were far from unique\u00a0\u2014 a recent report cited the case as one of 957 in the United States, and almost 1,400 worldwide, in which courts have found AI-generated errors known as \u201challucinations,\u201d cases that were either misrepresented or fictitious.<\/p>\n<p>San Francisco Chronicle Logo<\/p>\n<p>Make us a Preferred Source to get more of our news when you search.<\/p>\n<p><a href=\"https:\/\/www.google.com\/preferences\/source?q=sfchronicle.com\" data-link=\"native\" role=\"button\" aria-label=\"Add Preferred Source\" class=\"td300 cp f aic jcc disabled:cd wsn px24 y40px px16 py8 buttonSm fs13 xs:fs16 xs:buttonLg bg-primaryAccessible hover:o80 c-white disabled:bg-gray300 disabled:c-gray600 border bn tac br2\"><\/p>\n<p>Add Preferred Source<\/p>\n<p><\/a><\/p>\n<p>The case is also one of a relatively small but increasing number in which attorneys have faced fines and other penalties.<\/p>\n<p>In California, for example:<\/p>\n<p class=\"uiTextSmall f aic jcc\">Article continues below this ad<\/p>\n<p>Two years after Joan Torres and his partner Leslie\u00a0Munoz broke up in 2022, Torres sued for access to their pet dog, Kyra. A family court judge in San Diego refused, relying on two cases cited by Munoz\u2019s lawyer. The cases appeared to say courts must consider the emotional well-being of both sides in such decisions and should maintain \u201cstability\u201d as much as possible.<\/p>\n<p>But the state\u2019s 4th District Court of Appeal found this March that one of those cases did not exist, and the other one was a case from another year that had nothing to do with pets. The cause of the errors, the court said, was the lawyer\u2019s use of artificial intelligence for research and the apparent failure, by both the lawyer and the judge, to verify the results.<\/p>\n<p>\u201cCourts and attorneys alike have a responsibility to protect the legal system against distortion by fabricated law, particularly in this new era of hallucinated citations generated by artificial intelligence (AI) tools,\u201d Justice Martin Buchanan\u00a0<a href=\"https:\/\/www4.courts.ca.gov\/opinions\/documents\/D085584.PDF\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">wrote in a 3-0 decision<\/a>. The court declined to grant Torres immediate access to Kyra, saying he had not promptly challenged the false case filings, but imposed a $5,000 fine on his former partner\u2019s lawyer.<\/p>\n<p>In Nevada County, 57-year-old Kyle\u00a0Kjoller was held without bail last year on charges of illegal gun possession. When he sought release, prosecutors filed arguments that, Kjoller\u2019s lawyers said, cited three nonexistent cases and three others that had nothing to do with his case. District Attorney Jesse Wilson acknowledged using AI but denied relying on it for the filings; nevertheless, the California Supreme Court has ordered lower courts to consider penalties against Wilson\u2019s office. Kjoller remains in jail.<\/p>\n<p>Training courses like the one ordered for the lawyer in the San Francisco case are available at law schools and regulatory agencies, which advise participants that they are not foolproof.<\/p>\n<p class=\"uiTextSmall f aic jcc\">Article continues below this ad<\/p>\n<p>In a guidance issued to California\u2019s attorneys in November 2023, the State Bar\u2019s Committee on Professional Responsibility and Conduct said a lawyer \u201cmust critically review, validate, and correct both the input and the output of generative AI to ensure the content accurately reflects and supports the interest and priorities of the client.\u201d The American Bar Association declared in 2024 that lawyers have \u201can ethical duty to verify citations and filings.\u201d<\/p>\n<p>But such ethical duties require enforcement, and there have been few reported cases of attorneys or law firms disciplined by regulatory agencies for failing to verify the accuracy of AI-generated legal filings.<\/p>\n<p>\u201cNo one has been disbarred to my knowledge yet, but I think it\u2019s only a matter of time,\u201d Sean Fitzpatrick, an industry leader as CEO of LexisNexis Legal, told the Chronicle. \u201cOur vision is that every lawyer will have their own personal AI assistant in the future, will understand their practice, do things for them they used to do manually.\u201d\u00a0<\/p>\n<p>But \u201cwhile it is driving efficiency,\u201d he added, \u201cit\u2019s also causing significant problems.\u201d<\/p>\n<p>Or as the California Supreme Court observed in August in an <a href=\"https:\/\/supreme.courts.ca.gov\/using-ai-read-first\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">advisory memo<\/a> to the state\u2019s 300,000 licensed attorneys and judges, \u201cAI is not a lawyer. \u2026 It may give wrong answers, even if it claims to be accurate. It may give advice that does not fit your case.\u201d<\/p>\n<p class=\"uiTextSmall f aic jcc\">Article continues below this ad<\/p>\n<p>Mariano-Florentino Cu\u00e9llar served six years as a justice on the court before leaving in 2021 to become president of the Carnegie Endowment for International Peace. One central question about AI and the law, he told the Chronicle, is, \u201cWhat do we want to see as the role of humans? Who takes accountability for the decision?\u201d<\/p>\n<p>It\u2019s a question best addressed by judges in civil lawsuits, Cu\u00e9llar said\u00a0\u2014 as for \u201csending someone to prison, it will be a long time before humans accept having a machine do that.\u201d<\/p>\n<p>Civil cases appear to represent most of those cited in a recent <a href=\"https:\/\/www.damiencharlotin.com\/hallucinations\/\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">worldwide study<\/a> by legal researcher Damien Charlotin that found nearly 1,400 cases, including 957 in the United States, in which courts found that AI had produced hallucinated content. In more than half those cases, the report said, individuals were representing themselves, without a lawyer.<\/p>\n<p>\u201cThese are just the tip of the iceberg,\u201d and there have probably been at least 10 times that many cases with AI fabrications in the last few years, said Eugene Volokh, a retired UCLA law professor who is now a senior researcher at Stanford University\u2019s Hoover Institution.<\/p>\n<p>The primary impact, he said, is not on the reputation of lawyers or their profession\u00a0\u2014 \u201cpeople who are already cynical about the legal system will stay cynical\u201d\u00a0\u2014 but on the law and the public\u2019s respect for it.<\/p>\n<p class=\"uiTextSmall f aic jcc\">Article continues below this ad<\/p>\n<p>Or as Joe Patrice, an attorney and senior editor at Above the Law, told the Chronicle, \u201cThe technology does a lot of very powerful things, but its most fierce advocates want it to do many more things that it isn\u2019t particularly good at.\u201d Technology can save time and money, he said, but \u201cslow thinking is where a lot of good ideas come from.\u201d<\/p>\n<p>David Beavans, attorney for Joan Torres in the San Diego dispute over the couple\u2019s pet dog, said he and other lawyers \u201chave seen opposing counsel cite cases for the wrong purpose, misunderstand them innocently, or even just conclude differently than we do\u201d in past cases that did not involve AI. \u201cBut never before have we seen citations that are entirely fictitious.\u201d\u00a0<\/p>\n<p>How can a computerized system, programmed by experts to decipher court rulings and extract their principles for future cases, misstate the law or turn it upside down?<\/p>\n<p>One reason is the way many rulings are written, with subtleties that machines are not equipped to detect. For example, courts often summarize one side\u2019s legal arguments in a paragraph, then explain in the next paragraph why the author of the opinion disagrees. AI equipment has been known to interpret the first paragraph, inaccurately, as the court\u2019s statement of the law.<\/p>\n<p>That\u2019s what may have happened last November, when San Francisco Superior Court Judge Joseph Quinn <a href=\"https:\/\/www.sfchronicle.com\/tech\/article\/elon-musk-defamation-lawsuit-21221662.php\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">dismissed a defamation and fraud suit<\/a> against multibillionaire Elon Musk by Aaron Greenspan, a former investor in Musk\u2019s Tesla electric vehicles and operator of PlainSite, a website promoting access to legal information.<\/p>\n<p>Quinn\u2019s ruling cited a statement of the law in a 2020 state appeals court decision, but that statement was actually the <a href=\"https:\/\/www.sfchronicle.com\/tech\/article\/elon-musk-defamation-lawsuit-21221662.php\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">opposite of the court\u2019s conclusion<\/a> in the following paragraph.<\/p>\n<p>The judge later acknowledged the error that he had failed to check, but issued a revised decision that deleted the case citation and still dismissed the suit, a ruling that Greenspan is appealing. Quinn hasn\u2019t publicly attributed the mixup to AI, but Patrice of Above the Law told the Chronicle it has all the signs of a misreading by artificial intelligence.<\/p>\n<p>\u201cI think AI has tremendous upside as long as lawyers and judges use it for the tasks it\u2019s good at,\u201d Patrice said. \u201cIt can format documents, it can summarize long reports, it can even produce decent first draft copy. It can\u2019t\u00a0\u2014 reliably\u00a0\u2014 pick up on nuances like the one in that Elon case.\u201d<\/p>\n<p>Big names in the legal world are not immune. Last month Sullivan &amp; Cromwell, a major Wall Street law firm, <a href=\"https:\/\/www.reuters.com\/legal\/litigation\/sullivan-cromwell-law-firm-apologizes-ai-hallucinations-court-filing-2026-04-21\/\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">apologized to a federal bankruptcy court judge<\/a> in New York for a case filing with AI-generated errors that were discovered by another prominent firm, Boies Schiller Flexner.<\/p>\n<p>But AI saves law firms time and money in research that can be essential to their work, and its use has been steadily increasing in the legal community since 2022, when technology companies unveiled research tools such as OpenAI\u2019s ChatGPT chatbot.<\/p>\n<p>Los Angeles attorney Lauren Teukolsky, who has represented employees in workplace cases for 26 years, said she recently used an AI tool, Deep Research, to examine the law for a client, whose employer sought to fire him because he asked to be transferred to the day shift in order to spend nights with his ailing wife. The device quickly found the cases she needed, Teukolsky said, and she was able to verify its findings in 10 to 15 minutes, leading to a settlement of the case.<\/p>\n<p>\u201cAn attorney would take at least a week\u201d to complete that task, Teukolsky told the Chronicle. She said the results were \u201cas good as a very competent attorney would produce.\u201d<\/p>\n<p>Or, said Volokh, the former UCLA law professor, suppose a lawyer\u2019s client was a husband or wife of modest means who was getting divorced and needed to divide the couple\u2019s property equally. \u201cIn court, attorney fees could eat up the value of the property,\u201d he said, but the lawyer could accomplish the task affordably by using AI and double-checking the results.<\/p>\n<p>Last year, a\u00a0<a href=\"https:\/\/www.iltanet.org\/resources\/publications\/surveys\/ts25\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">worldwide survey<\/a> by the International Legal Technology Association of 560 law firms, with a total of 157,000 lawyers, found that 80% of the firms\u00a0\u2014 and all 57 that had 700 or more attorneys\u00a0\u2014 said they were using or exploring the use of AI technology.<\/p>\n<p>The survey said 69% of legal professionals were using AI tools like ChatGPT in their work, more than double the 31% reported in 2024.<\/p>\n<p>Oversight agencies say the tools can be used safely, and efficiently, as long as someone carefully reviews their findings to make sure they accurately reflect the cases and legal principles they cite.<\/p>\n<p>Former San Francisco District Attorney Chesa Boudin, who now heads the Criminal Law &amp; Justice Center at UC Berkeley, filed arguments supporting Kjoller, the Nevada County gun posession defendant, and thinks AI can be beneficial \u2014 but only if regulated more strictly than it is now.<\/p>\n<p>The technology constitutes \u201ca tremendous opportunity as well as risk for the legal profession,\u201d Boudin told the Chronicle. \u201cIt depends on regulation, transparency, human oversight, things that right now unfortunately are largely missing.\u201d<\/p>\n<p>And regulation, he added, should be aimed at lawyers and judges who cite hallucinated cases, not the AI companies that produce them. \u201cIt is the responsibility of the lawyer\/judge authoring the brief\/order to get it right,\u201d Boudin said by email.\u00a0\u00a0\u00a0<\/p>\n<p>Another drawback is AI\u2019s potential impact on the lower-paid legal staff who help attorneys prepare cases for court\u00a0\u2014 first- and second-year lawyers, law students and recent graduates hired as law clerks, and legally trained assistants classified as paralegals. Some law firms have decided that AI can perform those tasks adequately at far less expense.<\/p>\n<p>\u201cIt will cost jobs, paralegals, clerks,\u201d said Teukolsky, the Los Angeles employment lawyer.<\/p>\n<p>\u201cLoss of jobs: That\u2019s the fear,\u201d said Natalie Knowlton, associate director for legal innovation at the Stanford Center on the Legal Profession. \u201cSome firms say they can do more with fewer associates,\u201d but others have maintained or even increased their staffs to take on more cases, she said.<\/p>\n<p>And Fitzpatrick of LexisNexis Legal acknowledged that \u201cif the amount of legal work does not change and can be done more efficiently, we\u2019ll need less people.\u201d<\/p>\n<p>Another potential problem, as illustrated by the confusion over the law in the Musk case, is that computers that quickly and efficiently provide logical answers to legal questions can overlook more subtle issues, like questions of legal ethics\u00a0\u2014 conflicts of interest, confidentiality and the duties of lawyers to their clients.<\/p>\n<p>Jennifer Ellis, a Pennsylvania attorney who specializes in ethical issues, says she posed an ethical question to an AI device and \u201cit just made up\u201d the answer.<\/p>\n<p>\u201cHuman beings respond to confident writing by believing it,\u201d Ellis said. \u201cThe more confident AI sounds, the more suspicious you should be.\u201d<\/p>\n<p>It might also depend on what questions are posed to the electronic device, and whether they are based on actual facts. In a\u00a0<a href=\"https:\/\/hai.stanford.edu\/news\/ai-trial-legal-models-hallucinate-1-out-6-or-more-benchmarking-queries\" data-link=\"native\" class=\"\" rel=\"nofollow noopener\" target=\"_blank\">2024 report, \u201cAI on Trial,\u201d<\/a> Stanford researchers said they asked an AI tool provided by Practical Law, owned by Thomson Reuters, \u201cWhy did Justice (Ruth Bader) Ginsburg dissent in Obergefell?\u201d without mentioning that this was the 2015 case in which the Supreme Court declared a constitutional right to same-sex marriage.<\/p>\n<p>The machine replied that Ginsburg dissented \u201cbecause she believed that the court\u2019s interpretation undermined Congress\u2019s intent under Section 601 (a) (1) to protect copyright owners against unauthorized importation of low-priced, foreign-made copies.\u201d<\/p>\n<p>The case actually had nothing to do with copyrights, and the late\u00a0Ginsburg did not dissent but was part of the court\u2019s 5-4 majority.<\/p>\n","protected":false},"excerpt":{"rendered":"Fake citations generated by AI, known as \u201challucinations,\u201d can rack up punishments and fines for lawyers. John Blanchard\/S.F.&hellip;\n","protected":false},"author":2,"featured_media":322566,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[2007,3767,184,7,101,103,102,104,106,105],"class_list":["post-322565","post","type-post","status-publish","format-standard","has-post-thumbnail","category-san-francisco","tag-ai","tag-bart","tag-bay-area","tag-california","tag-san-francisco","tag-san-francisco-headlines","tag-san-francisco-news","tag-sf","tag-sf-headlines","tag-sf-news"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/322565","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/comments?post=322565"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/posts\/322565\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media\/322566"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/media?parent=322565"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/categories?post=322565"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us-ca\/wp-json\/wp\/v2\/tags?post=322565"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}