{"id":141872,"date":"2025-09-14T05:33:10","date_gmt":"2025-09-14T05:33:10","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/141872\/"},"modified":"2025-09-14T05:33:10","modified_gmt":"2025-09-14T05:33:10","slug":"ai-in-the-legal-system-from-hallucinations-to-high-court","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/141872\/","title":{"rendered":"AI in the legal system: From hallucinations to High Court"},"content":{"rendered":"

\u201cThe problem of hallucinations has not been solved,\u201d he said in an address to the Australian Bar Association<\/a> in late August.<\/a> \u201cFabricated case names, misquoted legislation and false principles are being put before courts.<\/a><\/p>\n

\u201cLawyers have been disciplined in most jurisdictions, including Australia, for relying on hallucinated AI content.<\/p>\n

\u2018Without transparency, accountability and oversight, these systems risk perpetuating the very injustices they aim to resolve.\u2019<\/p>\n

Matt McMillan, Lander & Rogers technology lawyer<\/p>\n

\u201cAI hallucinations remind us that accurate, reliable and critical legal analysis currently remains a solely human capability.\u201d<\/p>\n

And, just weeks ago, another high-profile case reverberated through the legal system. Submissions filed by a defendant in the Supreme Court of Victoria contained fabricated quotes and non-existent judgments generated by AI<\/a>, in a murder case. The defendant\u2019s barrister told Justice James Elliott he was \u201cdeeply sorry and embarrassed\u201d. The blunder delayed proceedings and led to a third round of corrected submissions.<\/p>\n

\u201cIt is not acceptable for AI to be used unless the product of that use is independently and thoroughly verified,\u201d Elliott told the Supreme Court in Melbourne.<\/p>\n

There have now been more than a dozen reported cases in Australian courts of documents containing false citations.<\/p>\n

Whether we\u2019re ready or not, generative AI has rapidly swept through Australia\u2019s workplaces and classrooms, and it\u2019s also entered our courtrooms.<\/p>\n

It raises the question: to what extent should our justice system allow AI, hallucinations at all? And should AI eventually wield the gavel?<\/p>\n

The prospect of an AI judge is no longer the stuff of science fiction. Estonia has already trialled an AI system to resolve small claims disputes under \u20ac7000 ($12000). Singapore\u2019s courts are experimenting with AI-assisted arbitration to unclog backlogs.<\/p>\n

At next month\u2019s SXSW Sydney, law firm Lander & Rogers will stage a provocative experiment: a fictional high-stakes embezzlement case set in 2035, in which an AI judge will weigh evidence, assess public sentiment using algorithms and deliver a sentence. The session will be led by Lander & Rogers executive Courtney Blackman, who last year ran a mock trial involving AI lawyers.<\/p>\n

\"At<\/p>\n

At last year\u2019s SXSW a mock trial pitted AI against human lawyers. From left: Jeanette Merjane, LawTech hub director Courtney Blackman, Ken Leung, Landers chief innovation officer Michelle Bey and UTS law professor David Lindsay.Credit: Louie Douvis<\/p>\n

Ellen Skinner is a sitting judge and is president of the Children\u2019s Court of New South Wales. She will join an Oxford-trained ethicist and a technology lawyer in debating AI\u2019s verdict at the end of the scenario. Lander & Rogers technology lawyer Matt McMillan is also sitting on the panel. He says AI systems have significant potential to enable greater access to justice, make legal processes more efficient, and reduce human bias.<\/p>\n

\"Lander<\/p>\n

Lander & Rogers technology lawyer Matt McMillan.<\/p>\n

With proper human oversight, AI can support a more equitable legal system, he says.<\/p>\n

\u201cThat human oversight is, however, essential to the issue of trust. Many current systems, for example, are trained on historical data which may reflect systemic biases. So, without transparency, accountability and oversight, these systems risk perpetuating the very injustices they aim to resolve.<\/p>\n

\u201cLegal reasoning, which often involves context and moral judgment, is complex, human, and cannot be fully captured by machines, no matter how intelligent they are. Although AI is smart, it remains, at least for now, an approximation of human thinking.\u201d<\/p>\n

For all the question marks around hallucinations, ethics and accuracy, AI tools have quickly become widely used by Australia\u2019s top-tier law offices and their clients.<\/p>\n

Silicon Valley legal AI start-up Harvey, co-founded by former Californian litigator Winston Weinberg, has signed up a third of Australia\u2019s top-tier firms, including King & Wood Mallesons, Ashurst, Gilbert + Tobin and Arnold Bloch Leibler, to its AI solution that provides instant analysis and data processing for lawyers. It has just opened a Sydney office, where it plans to hire 15 people.<\/p>\n

Weinberg says that the real potential for AI justice lies in low-stakes cases, at least for now. He thinks such a system could work in Australia.<\/p>\n

\"Harvey<\/p>\n

Harvey co-founders\u00a0Gabe Pereyra and Winston Weinberg.<\/p>\n

\u201cImagine two parties agreeing to submit a small claim to an AI arbitrator,\u201d he says. \u201cBoth sides put in their argument, the AI produces a ruling, and if either disagrees, they can appeal. That can speed up and unclog a lot of the legal system, but I think the reality is that for very high-stakes things you want a jury of your peers.\u201d<\/p>\n

It\u2019s a pragmatic vision: not robot judges in Supreme Court murder trials, but software resolving car park fender benders or landlord-tenant spats.<\/p>\n

\"There<\/p>\n

There have now been more than 20 reported cases in Australian courts of documents containing false citations.Credit: iStock<\/p>\n

England and Wales recently gave judges limited approval to use AI, though only for writing opinions, not for research or decision-making. Under that system, AI may help with the paperwork but it won\u2019t decide guilt or innocence.<\/p>\n

The Australian Institute for Judicial Administration, in a recent report, stressed that AI tools must meet the profession\u2019s highest standards of transparency and accountability. That means knowing how an algorithm reached its conclusion, which is something that remains a black box for most large language models.<\/p>\n

Proponents suggest AI could one day bring consistency to the bench. A University of Chicago study replicated a 2015 experiment involving 31 US federal judges, now comparing them with OpenAI\u2019s GPT-4o. Judges were swayed by emotional portrayals of defendants 65 per cent of the time, even when precedent dictated otherwise. AI, by contrast, remained unswayed and was driven purely by precedent. Researchers dubbed the AI a \u201cformalist judge\u201d, noting its approach is more akin to law students than seasoned jurists.<\/p>\n

Despite the risks, momentum is building and feels undeniable, at least for research and low-level tasks.<\/p>\n

Anthony Curtin, managing partner at Merton Lawyers, says that every time he interviews a graduate, lawyer or partner, he expects them to show a curiosity and willingness to adopt AI.<\/p>\n

Loading<\/p>\n

\u201cIt\u2019s going to revolutionise our industry, and if you\u2019re defensive on its application then you\u2019ll lose,\u201d he says.<\/p>\n

\u201cEvery lawyer and law student should be testing these platforms. They\u2019re not perfect yet, but their impact is already significant and accelerating. Without understanding this shift, you risk being behind the eight ball in recruitment and career progression.<\/p>\n

\u201cAs an industry we need to embrace and enjoy this. We are at a turning point in life, technology and how we operate. One day, we will look back with nostalgia and pride at being at the front row of this important shift.\u201d<\/p>\n

The Silicon Valley start-up Harvey claims its AI tech saves lawyers between 13 and 25 hours a month on research and drafting. For larger firms that integrate AI into custom workflows, the number can rise to more than 80 hours.<\/p>\n

At a time when governments are desperate to lift national productivity, those numbers are hard to ignore.<\/p>\n

Junior lawyers may spend less of their careers trawling through case law, and more time in court or advising clients, Weinberg says.<\/p>\n

The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"\u201cThe problem of hallucinations has not been solved,\u201d he said in an address to the Australian Bar Association…\n","protected":false},"author":2,"featured_media":141873,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[256,254,255,64,63,105],"class_list":{"0":"post-141872","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-au","12":"tag-australia","13":"tag-technology"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/141872","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=141872"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/141872\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/141873"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=141872"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=141872"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=141872"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}