When Judith Miller received the results of a medical imaging study last year, the 77-year-old Wisconsin resident did what many patients nowadays do: she asked AI to explain them. Claude, a large language model (LLM) developed by the company Anthropic, obligingly laid out possible interpretations. With the chatbot\u2019s analysis in hand, Miller went into her follow-up appointment feeling prepared for a productive conversation with her doctor. As she puts it, Claude\u2019s responses \u201cenabled me to better understand my health and engage more fully in shared decision-making.\u201d<\/p>\n
This scene has become commonplace in clinics around the country. Two recent polls both found that a third<\/a> of American adults<\/a> have turned to LLMs for health information\u2014to make sense of lab results, diagnose symptoms, research treatment options or inquire about prescription drugs. \u201cThe use of tools like these has doubled in the past year,\u201d says Robert Wachter, a physician at the University of California, San Francisco. \u201cI suspect they\u2019ll double again next year.\u201d<\/p>\n But these chatbots can also provide misleading or inaccurate advice, so experts urge caution when using them. Anthropic, for its part, agrees. “Claude is not designed or marketed for making clinical diagnoses,” according to a spokesperson for the company. Its proper use is “helping people prepare for conversations with their doctors, not replacing them.”<\/p>\n On supporting science journalism<\/p>\n If you’re enjoying this article, consider supporting our award-winning journalism by subscribing<\/a>. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today.<\/p>\n For many patients, AI is a welcome solution to the problems posed by the glut of personal health data supplied by the 21st Century Cures Act<\/a>, which mandates immediate online access to medical records, such as test results and clinical notes. \u201cIf you\u2019ve ever looked at that stuff,\u201d says Dave deBronkart, a health care blogger and activist, \u201cyou know it leaves you with the gigantic question: What does all this mean?\u201d Just a few years ago the meaning lay hidden behind a wall of medical jargon that only doctors could comprehend. And because patients can now view results online before speaking with a doctor, they\u2019re often left anxiously wondering what to make of it all. Today, however, general-purpose chatbots and a host of specialized health models can translate the jargon into plain language within seconds, potentially allaying unfounded fears.<\/p>\n Yet they may also heighten anxiety unnecessarily\u2014or worse. LLMs remain prone to mistakes. They can present falsehoods as facts and sycophantically reinforce<\/a> users\u2019 prior (and sometimes misguided) beliefs. Though these character flaws may lessen as the models grow more powerful, many experts express concern about the potential risks of using today\u2019s AI models in this way. \u201cThere aren\u2019t a lot of guardrails around breaking them, pushing them to tell you actual misinformation,\u201d says Cait DesRoches, executive director of OpenNotes, a nonprofit that promotes patients\u2019 access to medical records. She adds that there is little research on what happens when people treat an LLM as a health authority: \u201cI don\u2019t think we have any idea how well it works for average patients.\u201d<\/p>\n Worst-case scenarios have already surfaced. In December a 75-year-old Seattle man died of a treatable type of leukemia; he reportedly refused treatment on the basis of AI-generated evidence<\/a> that incorrectly suggested he had a rare complication. Some of the preliminary research on how people use AI for medical diagnosis is sobering. In a Nature Medicine<\/a> study published in February, researchers asked participants to diagnose a hypothetical condition with the help of various LLMs. They reached the right conclusion only about a third of the time.<\/p>\n Still, most experts agree that chatbots can be helpful to people seeking medical information, if used cautiously. \u201cI don\u2019t think people should avoid using them,\u201d DesRoches says, \u201cbut I do think people should use them with their eyes open.\u201d Adam Rodman, a general internist at Beth Israel Deaconess Medical Center, goes even further: \u201cI would argue that LLMs, if used appropriately\u2014that\u2019s a big caveat\u2014are the best tool for patient empowerment ever invented.\u201d<\/p>\n Hoping to harness this technology without compromising safety, researchers have developed a suite of strategies to counteract AI\u2019s shortcomings. For example, they suggest telling chatbots to take on the persona of a doctor. This may \u201cprompt the model to collect data in a physicianlike manner,\u201d Rodman says. Other tactics include asking an LLM to rigorously reevaluate its own reasoning and seeking a \u201csecond opinion\u201d from a different model. Rodman stresses the importance of removing personal information, such as your name and Social Security number, from any chatbot input to protect your privacy.<\/p>\n Ideally, after all that digital dialogue, patients would wind up with better-informed questions for their doctors. Wachter describes this trend as \u201cgenerally healthy,\u201d though he sometimes loses valuable time debunking Dr. Chatbot\u2019s faulty advice. \u201cI\u2019ve got 15 minutes for this appointment,\u201d he says, \u201cand I\u2019m going to have to spend the first 10 minutes talking the patient down from what GPT told them to do.\u201d<\/p>\n In many cases LLMs are likely replacing real-life clinical advice altogether, particularly for those who are uninsured or face long wait times to get an appointment. \u201cThe access issue is at crisis level,\u201d says Laura Adams, a senior adviser to the National Academy of Medicine on AI matters. Despite the technology\u2019s limitations, she argues we must compare it not to perfection but to reality, in which the alternative may be no care at all. \u201cIt\u2019s better than nothing,\u201d she says.<\/p>\n With AI and medical advice, Adams notes that \u201cthe horse is way out of the barn.\u201d As more people lean on chatbots to manage their health, researchers and patient advocates say this moment demands a new form of AI literacy. \u201cThe remedy is not to keep people ignorant,\u201d deBronkart says. \u201cIt\u2019s to teach them how to do it better\u201d by educating children and adults alike. On top of that, newer LLMs will likely improve in medical uses\u2014Wachter suggests that some models might eventually undergo board certification, as actual physicians do.<\/p>\n For now, people like Miller are already approaching AI just as DesRoches recommends: with eyes open, aware of its tendency to hallucinate and confirm user biases. Sophisticated as chatbots\u2019 responses may be, they are stitched together from statistical patterns in large datasets\u2014an impressive trick but one that still falls short of the breadth and reliability in human-level clinical reasoning. \u201cIt\u2019s just following up words that were probable,\u201d Miller says. \u201cI\u2019m not looking at it as a source of absolute truth.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"When Judith Miller received the results of a medical imaging study last year, the 77-year-old Wisconsin resident did…\n","protected":false},"author":2,"featured_media":673257,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[64,63,137,500],"class_list":["post-673256","post","type-post","status-publish","format-standard","has-post-thumbnail","category-healthcare","tag-au","tag-australia","tag-health","tag-healthcare"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/673256","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=673256"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/673256\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/673257"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=673256"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=673256"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=673256"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}