Earlier this month, an appeals court in California issued a blistering decision and record $10,000 fine against a lawyer who submitted a brief in which \u201cnearly all of the legal quotations in plaintiff\u2019s opening brief, and many of the quotations in plaintiff\u2019s reply brief, are fabricated\u201d through the use of ChatGPT, Claude, Gemini, and Grok. The court said it was publishing its opinion \u201cas a warning\u201d to California lawyers that they will be held responsible if they do not catch AI hallucinations in their briefs.\u00a0<\/p>\n
In that case<\/a>, the lawyer in question \u201casserted that he had not been aware that generative AI frequently fabricates or hallucinates legal sources and, thus, he did not \u2018manually verify [the quotations] against more reliable sources.\u2019 He accepted responsibility for the fabrications and said he had since taken measures to educate himself so that he does not repeat such errors in the future.\u201d<\/p>\n As the judges remark in their opinion, the use of generative AI by lawyers is now everywhere, and when it is used in ways that introduce fake citations or fake evidence, it is bogging down courts all over America (and the world). For the last few months, 404 Media has been analyzing dozens of court cases around the country in which lawyers have been caught using generative AI to craft their arguments, generate fictitious citations, generate false evidence, cite real cases but misinterpret them, or otherwise take shortcuts that has introduced inaccuracies into their cases. Our main goal was to learn more about why lawyers were using AI to write their briefs, especially when so many lawyers have been caught making errors that lead to sanctions and that ultimately threaten their careers and their standings in the profession.<\/p>\n To do this, we used a crowdsourced database of AI hallucination cases maintained by the researcher Damien Charlotin, which so far contains more than 410 cases worldwide<\/a>, including 269 in the United States. Charlotin\u2019s database is an incredible resource, but it largely focuses on what happened in any individual case and the sanctions against lawyers, rather than the often elaborate excuses that lawyers told the court when they were caught. Using Charlotin\u2019s database as a starting point, we then pulled court records from around the country for dozens of cases where a lawyer offered a formal explanation or apology. Pulling this information required navigating clunky federal and state court record systems and finding and purchasing the specific record where the lawyer in question tried to explain themselves (these were often called \u201cresponses to order to show cause.\u201d) We also reached out to lawyers who were sanctioned for using AI to ask them why they did it. Very few of them responded, but we have included explanations from the few who did.\u00a0<\/p>\n What we found was incredibly fascinating, and reveals a mix of lawyers blaming IT issues, personal and family emergencies, their own poor judgment and carelessness, and demands from their firms and the industry to be more productive and take on more casework. But most often, they simply blame their assistants.\u00a0<\/p>\n Few dispute that the legal industry is under great pressure to use AI. Legal giants like Westlaw and LexisNexis have pitched bespoke tools to law firms that are now regularly being used, but Charlotin\u2019s database makes clear that lawyers are regularly using off-the-shelf generalized tools like ChatGPT and Gemini as well. There\u2019s a seemingly endless number of startups selling AI legal tools that do research, write briefs, and perform other legal tasks. While working on this article, it became nearly impossible to keep up with new cases of lawyers being sanctioned for using AI. Charlotin has documented 11 new cases within the last week alone.\u00a0<\/p>\n This article is the first of several 404 Media will write exploring the use of AI in the legal profession. If you\u2019re a lawyer and have thoughts or firsthand experiences, please get in touch. Some of the following anecdotes have been lightly edited for clarity.<\/p>\n \ud83d\udca1<\/p>\n Are you a lawyer or do you work in the legal industry? We want to know how AI is impacting the industry, your firm, and your job. Using a non-work device, you can message me securely on Signal at jason.404. Otherwise, send me an email at jason@404media.co.<\/p>\n A lawyer in Indiana blames the court (Fake case cited)<\/p>\n A judge stated that the lawyer<\/a> \u201ctook the position that the main reason for the errors in his brief was the short deadline (three days) he was given to file it. He explained that, due to the short timeframe and his busy schedule, he asked his paralegal (who once was, but is not currently, a licensed attorney) to draft the brief, and did not have time to carefully review the paralegal’s draft before filing it.\u201d<\/p>\n A lawyer in New York<\/a> blamed vertigo, head colds, and malware<\/p>\n “He acknowledges that he used Westlaw supported by Google Co-Pilot which is an artificial intelligence-based tool as preliminary research aid.” The lawyer \u201cgoes on to state that he had no idea that such tools could fabricate cases but acknowledges that he later came to find out the limitation of such tools. He apologized for his failure to identify the errors in his affirmation, but partly blames \u2018a serious health challenge since the beginning of this year which has proven very persistent which most of the time leaves me internally cold, and unable to maintain a steady body temperature which causes me to be dizzy and experience bouts of vertigo and confusion.\u2019 The lawyer then indicates that after finding about the \u2018citation errors\u2019 in his affirmation, he conducted a review of his office computer system and found out that his system was \u2018affected by malware and unauthorized remote access.\u2019 He says that he compared the affirmation he prepared on April 9, 2025, to the affirmation he filed to [the court] on April 21, 2025, and \u2018was shocked that the cases I cited were substantially different.\u2019\u201d<\/p>\n A lawyer in Florida blames a paralegal and the fact they were doing the case pro bono (Fake cases and hallucinated quotes)<\/p>\n The lawyer<\/a> \u201cexplained that he was handling this appeal pro bono and that as he began preparing the brief, he recognized that he lacked experience in appellate law. He stated that at his own expense, he hired \u2018an independent contractor paralegal to assist in drafting the answer brief.\u2019 He further explained that upon receipt of a draft brief from the paralegal, he read it, finalized it, and filed it with this court. He admitted that he \u2018did not review the authority cited within the draft answer brief prior to filing\u2019 and did not realize it contained AI generated content.<\/p>\n A lawyer in South Carolina said he was rushing (Fake cases generated by Microsoft CoPilot)<\/p>\n \u201cOut of haste and a na\u00efve understanding of the technology, he did not independently verify the sources were real before including the citations in the motion filed with the Court seeking a preliminary injunction\u201d<\/p>\n A lawyer in Hawaii blames a New Yorker they hired<\/p>\n This lawyer was sanctioned $100 by a court for one AI-generated case, as well as quoting multiple real cases and misattributing them to that fake case. They said they had hired a per-diem attorney\u2014\u201csomeone I had previously worked with and trusted,\u201d they told the court\u2014to draft the case, and though they \u201cdid not personally use AI in this case, I failed to ensure every citation was accurate before filing the brief.\u201d The Honolulu Civil Beat reported<\/a> that the per-diem attorney they hired was from New York, and that they weren\u2019t sure if that attorney had used AI or not.\u00a0<\/p>\n The lawyer told us over the phone that the news of their $100 sanction had blown up in their district thanks to that article. \u201c\u200aI was in court yesterday, and of course the [opposing] attorney somehow brought this up,\u201d they said in a call. According to them, that attorney has also used AI in at least seven cases. Nearly every lawyer is using AI to some degree, they said; it\u2019s just a problem if they get caught. \u201cThe judges here have seen it extensively. I know for a fact other attorneys have been sanctioned. It\u2019s public, but unless you know what to search for, you\u2019re not going to find it anywhere. It\u2019s just that for some stupid reason, my matter caught the attention of a news outlet. It doesn\u2019t help with business.\u201d<\/p>\n A lawyer in Arizona blames someone they hired<\/p>\n A judge wrote \u201cthis is a case where the majority of authorities cited were either fabricated, misleading, or unsupported. That is egregious \u2026 this entire litigation has been derailed by Counsel\u2019s actions. The Opening Brief was replete with citation-related deficiencies, including those consistent with artificial intelligence generated hallucinations.\u201d<\/p>\n