{"id":649548,"date":"2026-05-05T15:16:19","date_gmt":"2026-05-05T15:16:19","guid":{"rendered":"https:\/\/www.newsbeep.com\/ca\/649548\/"},"modified":"2026-05-05T15:16:19","modified_gmt":"2026-05-05T15:16:19","slug":"ai-hallucinations-risk-undermining-legal-system-research","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ca\/649548\/","title":{"rendered":"AI &#8216;hallucinations&#8217; risk undermining legal system: research"},"content":{"rendered":"<p>As artificial intelligence becomes more widely used across Quebec\u2019s legal profession, a McGill researcher warns AI risks eroding trust in the legal system.<\/p>\n<p>Problems are already cropping up in courtrooms, with judges in recent months finding cases of AI-generated fabrications.<\/p>\n<p>AI \u201challucinations,\u201d where generative AI tools spit out fictional information, are \u201cfundamentally a part of how these models work,\u201d McGill law professor Jennifer Raso said.<\/p>\n<p>AI tools in courtrooms \u201crisk not just misrepresenting things, but actually creating significant injustices,\u201d she said.<\/p>\n<p>In recent rulings, Quebec judges have found generative AI \u201challucinations\u201d in documents, in some cases imposing penalties for the use of non-existent legal references.<\/p>\n<p>A judgment signed in April found that Michel A. Jeanniot, a Montreal arbitrator, had <a href=\"https:\/\/montrealgazette.com\/news\/montreal-arbitrator-relied-on-ai-hallucinations-to-reach-decision-in-health-care-dispute\/\" type=\"link\" id=\"https:\/\/montrealgazette.com\/news\/montreal-arbitrator-relied-on-ai-hallucinations-to-reach-decision-in-health-care-dispute\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">cited non-existent court decisions<\/a> \u201challucinated\u201d by AI.<\/p>\n<p>Last December, an applicant who was representing herself was found to have referenced non-existent decisions in her filing, which the judge said were probably AI hallucinations. Superior Court Judge Catherine Dagenais fined her $500 for filing a document that \u201ccontained fictional jurisprudential references.\u201d<\/p>\n<p>And in October, Superior Court Judge Luc Morin fined Jean Laprade $5,000 for including AI-hallucinated decisions in his legal defence.<\/p>\n<p>It is the very design of AI tools that favour hallucinations, Raso said, meaning advances in the technology are unlikely to resolve the issue.<\/p>\n<p>\u201cThese models are designed to use very complex statistical analysis mechanisms to basically predict\u201d a response, she said. \u201cTruth is not part of how the model is designed.\u201d<\/p>\n<p>The result can be \u201ca citation of a case that doesn\u2019t exist,\u201d she said, or \u201ca provision of a statute that doesn\u2019t exist.\u201d<\/p>\n<p>AI tools are mostly trained in English, Raso said, which makes them more error-prone in other languages. <\/p>\n<p>\u201cIn places like Quebec, where the decisions are made in French, there\u2019s a higher risk of error.\u201d<\/p>\n<p>The Barreau du Qu\u00e9bec, which represents Quebec\u2019s lawyers, says it\u2019s taking actions to ensure its members use AI ethically.<\/p>\n<p>\u201cThe Barreau is the first professional order to require that all its members take a course on the guidelines for ethical use of artificial intelligence,\u201d Barreau director Catherine Ouimet said in a statement.<\/p>\n<p>Training lawyers \u201cis a good first step,\u201d Raso said, \u201cbut the danger is assuming that AI training will do all the work needed.\u201d<\/p>\n<p>Many lawyers are feeling pressure to use AI tools, she said, but don\u2019t question how the tools work and what their shortfalls may be.<\/p>\n<p>The April decision that found arbitrator Jeanniot had cited non-existent court decisions in his ruling was particularly alarming, she said, because it showed that he had abdicated his duty to make a decision himself.<\/p>\n<p>Arbitrators \u201chave a duty to hear the sides, to reflect on the evidence that\u2019s been provided to them, to think about the law and make an expert judgment.\u201d<\/p>\n<p>\u201cWe can\u2019t just offload this to an algorithmic decision-making system,\u201d she said. \u201cIt\u2019s inappropriate on so many levels.\u201d<\/p>\n<p>Ouimet said the Barreau couldn\u2019t comment on a specific case, so wouldn\u2019t confirm whether or not the association was investigating Jeanniot or other AI-related incidents.<\/p>\n<p>But she said the organization \u201ccan engage the various mechanisms at its disposal\u201d in cases of ethical breaches, including penalties such as fines, limits on a lawyer\u2019s practice and disbarment.<\/p>\n<p>Raso said she expected Jeanniot would face some sort of consequence, such as a fine, as well as reputational damage likely to hurt his chances at landing future arbitration gigs.<\/p>\n<p>Although the arbitrator\u2019s decision was challenged in court, Raso said other decision makers, such as bureaucrats handling immigration and social insurance files, face less legal scrutiny.<\/p>\n<p>\u201cIt\u2019s a real danger\u201d that AI tools could influence a wide range of decisions, she said.<\/p>\n<p>If \u201cdecisions are being generated using tools that are unreliable \u2026 that raises fundamental problems for the public\u2019s faith in the justice system,\u201d Raso said.<\/p>\n<p>\u201cIf going to an arbitrator or tribunal member is as reliable as shaking a Magic 8 Ball and getting a result, then why would we even go there? This is a recipe for disaster,\u201d she said.<\/p>\n<p>\t\t\t\t\t\t\t<a href=\"https:\/\/montrealgazette.com\/author\/jawilson\/\" class=\"postmedia-author-card__image-link\" rel=\"author image nofollow noopener\" target=\"_blank\"><img decoding=\"async\" alt=\"Profile picture of Jack Wilson\" src=\"https:\/\/www.newsbeep.com\/ca\/wp-content\/uploads\/2026\/04\/jack-wilson-2-e1774792320471.jpg\" class=\"avatar avatar-96 photo\" height=\"96\" width=\"96\" loading=\"lazy\"\/><\/a><\/p>\n<p>\n\t\t\t\t\tI&#8217;ve reported for The Gazette since 2024. I love covering a good politics story, but any day I tell Montrealers something new about the world around them feels like a good one. Got a tip for me? Email me at jawilson@postmedia.com\t\t\t\t<\/p>\n","protected":false},"excerpt":{"rendered":"As artificial intelligence becomes more widely used across Quebec\u2019s legal profession, a McGill researcher warns AI risks eroding&hellip;\n","protected":false},"author":2,"featured_media":649549,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[62,276,277,49,48,222407,61],"class_list":["post-649548","post","type-post","status-publish","format-standard","has-post-thumbnail","category-artificial-intelligence","tag-ai","tag-artificial-intelligence","tag-artificialintelligence","tag-ca","tag-canada","tag-crime-and-law","tag-technology"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/649548","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/comments?post=649548"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/posts\/649548\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media\/649549"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/media?parent=649548"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/categories?post=649548"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ca\/wp-json\/wp\/v2\/tags?post=649548"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}