Dealing with a loved one\u2019s belongings after their death is never easy. But as Alaska\u2019s state courts have discovered, an inaccurate or misleading artificial intelligence chatbot can easily make matters worse.<\/p>\n
For more than a year, Alaska\u2019s court system has been designing a pioneering generative AI chatbot termed the Alaska Virtual Assistant (AVA) to help residents navigate the tangled web of forms and procedures involved in probate, the judicial process of transferring property away from a deceased person.<\/p>\n
Yet what was meant to be a quick, AI-powered leap forward in increasing access to justice has spiraled into a protracted, yearlong journey plagued by false starts and false answers.<\/p>\n
AVA \u201cwas supposed to be a three-month project,\u201d said Aubrie Souza, a consultant with the National Center for State Courts<\/a> (NCSC) who has worked on and witnessed AVA\u2019s evolution. \u201cWe are now at well over a year and three months, but that\u2019s all because of the due diligence that was required to get it right.\u201d<\/p>\n Designing this bespoke AI solution has illuminated the difficulties government agencies across the United States are facing in applying powerful AI systems to real-world problems where truth and reliability are paramount.<\/p>\n \u201cWith a project like this, we need to be 100% accurate, and that\u2019s really difficult with this technology,\u201d said Stacey Marz, the administrative director of the Alaska Court System<\/a> and one of the AVA project\u2019s leaders.<\/p>\n \u201cI joke with my staff on other technology projects that we can\u2019t expect these systems to be perfect, otherwise we\u2019d never be able to roll them out. Once we get the minimum viable product, let\u2019s get that out there, and then we\u2019ll enhance that as we learn.\u201d<\/p>\n But Marz said she thinks this chatbot should be held to a higher standard. \u201cIf people are going to take the information they get from their prompt and they\u2019re going to act on it and it\u2019s not accurate or not complete, they really could suffer harm. It could be incredibly damaging to that person, family or estate.\u201d<\/p>\n While many local government agencies are experimenting with AI tools for use cases ranging from helping residents apply for a driver\u2019s license<\/a> to speeding up municipal employees\u2019 ability to process housing benefits<\/a>, a recent Deloitte report<\/a> found that less than 6% of local government practitioners were prioritizing AI as a tool to deliver services.<\/p>\n The AVA experience demonstrates the barriers government agencies face in attempting to leverage AI for increased efficiency or better service, including concerns about reliability and trustworthiness in high-stakes contexts, along with questions about the role of human oversight given fast-changing AI systems. These limitations clash with today\u2019s rampant AI hype<\/a> and could help explain larger discrepancies between booming AI investment<\/a> and limited AI adoption<\/a>.<\/p>\n Marz envisioned the AVA project as a cutting-edge, low-cost version of Alaska\u2019s family law helpline<\/a>, which is staffed by court employees and provides free guidance about legal matters ranging from divorce to domestic violence protective orders.<\/p>\n \u201cOur goal was to basically try to replicate the services with the chatbot that we would provide with a human facilitator,\u201d Marz told NBC News, referring to AVA\u2019s team of attorneys, technical experts and advisers from the NCSC. \u201cWe wanted a similar self-help experience, if somebody was able to talk to you and say, \u2018This is what I need help with, this is my situation.\u2019\u201d<\/p>\n While the NCSC provided an initial grant to get AVA off the ground as part of its growing work on AI<\/a>, the chatbot has been technically developed by Tom Martin, a lawyer and law professor who launched a law-focused AI company called LawDroid<\/a> and designs legal AI tools.<\/p>\n Describing the AVA service, Martin highlighted many critical decisions and considerations that go into the design process, such as choosing and shaping an AI system\u2019s personality.<\/p>\n Many commentators<\/a> and researchers<\/a> have illustrated how certain models or versions of AI systems behave in different ways, almost as if they adopt different personas. Researchers and even users can alter these personas<\/a> through technical tweaks, as many ChatGPT users found out earlier this year<\/a> when the OpenAI service fluctuated between personalities that were either gushing and sycophantic or emotionally distant. Other models, like xAI\u2019s Grok, are known for having looser guardrails and increased willingness to embrace controversial topics<\/a>.<\/p>\n \u201cDifferent models have almost different types of personalities,\u201d Martin told NBC News. \u201cSome of them are very good at rule-following, while others are not as good at following rules and kind of want to prove that they\u2019re the smartest guy in the room.\u201d <\/p>\n \u201cFor a legal application, you don\u2019t want that,\u201d Martin said. \u201cYou want it to be rule-following but smart and able to explain itself in plain language.\u201d<\/p>\n Even traits that would otherwise be welcomed become more problematic when applied to topics as consequential as probate. Working with Martin, NCSC\u2019s Souza noted that early versions of AVA were too empathetic and annoyed users who might have been actively grieving and simply wanted answers about the probate process: \u201cThrough our user testing, everyone said, \u2018I\u2019m tired of everybody in my life telling me that they\u2019re sorry for my loss.\u2019\u201d <\/p>\n \u201cSo we basically removed those kinds of condolences, because from an AI chatbot, you don\u2019t need one more,\u201d Souza said.<\/p>\n Beyond the system\u2019s superficial tone and pleasantries, Martin and Souza had to contend with the serious issue of hallucinations<\/a>, or instances in which AI systems confidently share false or exaggerated information.<\/p>\n \u201cWe had trouble with hallucinations, regardless of the model, where the chatbot was not supposed to actually use anything outside of its knowledge base,\u201d Souza told NBC News. \u201cFor example, when we asked it, \u2018Where do I get legal help?\u2019 it would tell you, \u2018There\u2019s a law school in Alaska, and so look at the alumni network.\u2019 But there is no law school in Alaska.\u201d<\/p>\n Martin has worked extensively to ensure the chatbot only references the relevant areas of the Alaska Court System\u2019s probate documents rather than conducting wider web searches.<\/p>\n Across the AI industry, AI hallucinations have decreased over time<\/a> and present less of a threat today than they did even several months ago. Many companies building AI applications like AI-agent provider Manus, which was recently acquired by Meta for more than $2 billion<\/a>, stress the reliability of their services and include several layers of AI-powered verification to ensure their results are accurate.<\/p>\n To evaluate the accuracy and helpfulness of AVA\u2019s responses, the AVA team designed a set of 91 questions regarding probate topics, asking the chatbot, for example, which probate form would be appropriate to submit if a user wanted to transfer the title of their deceased relative\u2019s car to their name.<\/p>\n Yet the 91-question test proved too time-consuming to run and evaluate, according to Jeannie Sato, the Alaska Court System\u2019s director of access to justice services, given the stakes at hand and the need for human review.<\/p>\n So Sato said the team landed on a refined list of just 16 test questions, featuring \u201csome questions that AVA had answered incorrectly, some that were complicated, and some that were pretty basic questions that we think AVA may be asked frequently.\u201d<\/p>\n Cost is another critical issue for Sato and the AVA team. New iterations and versions of AI systems have caused usage fees to fall precipitously<\/a>, which the AVA team sees as a key advantage of AI tools given limited court budgets.<\/p>\n Martin told NBC News that under one technical setup, 20 AVA queries would cost only about 11 cents. \u201cI\u2019m mission-driven, and it\u2019s about impact for me in helping people in the world,\u201d Martin said. \u201cTo be able to carry forward with that mission, of course, cost is extremely important.\u201d<\/p>\n