{"id":203345,"date":"2025-10-05T12:54:08","date_gmt":"2025-10-05T12:54:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/us\/203345\/"},"modified":"2025-10-05T12:54:08","modified_gmt":"2025-10-05T12:54:08","slug":"chatbot-dreams-generate-ai-nightmares-for-bay-area-lawyers","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/us\/203345\/","title":{"rendered":"Chatbot dreams generate AI nightmares for Bay Area lawyers"},"content":{"rendered":"

A Palo Alto lawyer with nearly a half-century of experience admitted to an Oakland federal judge this summer that legal cases he referenced in an important court filing didn\u2019t actually exist and appeared to be products of artificial intelligence \u201challucinations.\u201d<\/a><\/p>\n

Jack Russo, in a court filing, described the apparent AI fabrications as a \u201cfirst-time situation\u201d for him and added, \u201cI am quite embarrassed about it.\u201d<\/p>\n

A specialist in computer law, Russo found himself in the rapidly growing company of lawyers publicly shamed as wildly popular but error-prone\u00a0artificial intelligence technology like ChatGPT<\/a> collides with the rigid rules of legal procedure.<\/p>\n

Hallucinations \u2014 when AI produces inaccurate or nonsensical information \u2014 have posed an ongoing problem in the generative AI<\/a> that has birthed a Silicon Valley frenzy<\/a> since San Francisco\u2019s OpenAI released its ChatGPT bot in late 2022. In the legal arena, AI-generated errors are drawing heightened scrutiny as lawyers flock to the technology, and irate judges are making referrals to disciplinary authorities and, in dozens of U.S. cases since 2023, levying financial penalties of up to $31,000, including a California-record fine of $10,000 last month<\/a> in a Southern California case.<\/p>\n

Chatbots respond to users\u2019 prompts by drawing on vast troves of data and use pattern analysis and sophisticated guesswork to produce results. Errors can occur for many reasons, including insufficient or flawed AI-training data or incorrect assumptions by the AI. It affects not just lawyers, but ordinary people seeking information, as when Google\u2019s AI overviews last year told users to eat rocks, and add glue to pizza sauce<\/a> to keep the cheese from sliding off.<\/p>\n

Russo told Judge Jeffrey White he took full responsibility for not ensuring the filing was factual, but said a long recovery from COVID at an age beyond 70 led him to delegate tasks to support staff without \u201cadequate supervision protocols\u201d in place.<\/p>\n

\"EricSanta Clara University law professor Eric Goldman (Josie Lepe\/Bay Area News Group)\u00a0<\/p>\n

\u201cNo sympathy here,\u201d internet law professor Eric Goldman of Santa Clara University said. \u201cEvery lawyer can tell a sob story, but I\u2019m not falling for it. We have rules that require lawyers to double-check what they file.\u201d<\/p>\n

The judge wrote in a court order last month that Russo\u2019s AI-dreamed fabrications were a first for him, too. Russo broke a federal court rule by failing to adequately check his motion to throw out a contract dispute case, White wrote. The court, the judge sniped, \u201chas been required to divert its attention from the merits of this and other cases to address this issue.\u201d<\/p>\n

White issued a preliminary order requiring Russo to pay some of the opposing side\u2019s legal fees. Russo told White his firm, Computerlaw Group, had \u201ctaken steps to fix and prevent a reoccurrence.\u201d Russo declined to answer questions from this news organization.<\/p>\n

As recently as mid-2023 it was a novelty to find a lawyer facing a reprimand for submitting court filings referring to nonexistent cases conjured up by artificial intelligence<\/a>, but now such incidents arise nearly by the day, and even judges have been implicated, according to a database compiled by Damien Charlotin, a senior fellow at French business school HEC Paris who is tracking worldwide legal filings containing AI hallucinations.<\/p>\n

\u201cI think the acceleration is still ongoing,\u201d Charlotin said.<\/p>\n

Charlotin said his database<\/a> includes \u201ca surprising number\u201d of lawyers who are sloppy, reckless or \u201cplain bad.\u201d<\/p>\n

In May, San Francisco lawyer Ivana Dukanovic admitted in U.S. District Court in San Jose to an \u201cembarrassing and unintentional mistake\u201d by herself and others at legal firm Latham & Watkins. While representing San Francisco AI giant Anthropic in a music copyright case, they submitted a filing with hallucinated material, Dukanovic wrote. Dukanovic \u2014 whose company bio lists \u201cartificial intelligence\u201d as one of her areas of legal practice \u2014 blamed the creation of the false information on a particular chatbot: Claude.ai, the flagship product of her client Anthropic.<\/p>\n

Judge Susan van Keulen ordered part of the filing removed from the court record. Dukanovic, who, along with her firm, appears to have dodged sanctions, did not respond to requests for comment.<\/p>\n

Charlotin has found 113 U.S. cases involving lawyers submitting filings with hallucinated material, mostly legal-case citations, that have been the subject of court decisions since mid-2023. He believes many court submissions with AI fabrications are never caught, potentially affecting case outcomes.<\/p>\n

Court decisions can have \u201clife-changing consequences,\u201d including in matters involving child custody or disability claims, law professor Goldman said.<\/p>\n

\u201cThe stakes in some cases are so high, and if someone is distorting the judge\u2019s decision-making, the system breaks down,\u201d Goldman said.<\/p>\n

Still, AI can be a useful tool for lawyers, finding information people might miss, and helping to prepare documents, he said. \u201cIf people use AI wisely, it helps them do a better job,\u201d Goldman said. \u201cThat\u2019s pushing everyone to adopt it.\u201d<\/p>\n

Survey results released in April by the American Bar Association<\/a>, the nation\u2019s largest lawyers group, found that AI use by law firms almost tripled last year to 30% of responding law offices from 11% in 2023<\/a>, and that ChatGPT was the \u201cclear leader across firms of every size.\u201d<\/p>\n

Fines may be the least of a lawyer\u2019s worries, Goldman said. A judge could flag an attorney to their licensing organization for discipline, or dismiss a case, or reject a key filing, or view everything the lawyer does in the case with skepticism. A client could sue for malpractice. Orders to pay the other side\u2019s legal fees can require six-figure payments.<\/p>\n

Charlotin\u2019s database shows judges slapping many lawyers with warnings or referrals to disciplinary authorities, and sometimes purging all or part of a filing from the court record, or ordering payment of the opposition\u2019s fees. Last year, a federal appeals court in California threw out an appeal it said was \u201creplete with misrepresentations and fabricated case law,\u201d including \u201ctwo cases that do not appear to exist.\u201d<\/p>\n

Charlotin expects his database to keep swelling.<\/p>\n

\u201cI don\u2019t really see it decrease on the expected lines of \u2018surely everyone should know by now,’\u201d Charlotin said.<\/p>\n","protected":false},"excerpt":{"rendered":"A Palo Alto lawyer with nearly a half-century of experience admitted to an Oakland federal judge this summer…\n","protected":false},"author":2,"featured_media":203346,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45],"tags":[182,181,507,3805,28,2356,2302,2558,983,10432,10836,5284,10838,71080,10840,10843,74],"class_list":["post-203345","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-bay-area","tag-business","tag-california","tag-courts","tag-latest-headlines","tag-local-news","tag-palo-alto","tag-peninsula","tag-san-francisco","tag-san-francisco-county","tag-santa-clara","tag-santa-clara-county","tag-south-bay","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/203345","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/comments?post=203345"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/posts\/203345\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media\/203346"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/media?parent=203345"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/categories?post=203345"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/us\/wp-json\/wp\/v2\/tags?post=203345"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}