{"id":49158,"date":"2025-09-28T17:55:18","date_gmt":"2025-09-28T17:55:18","guid":{"rendered":"https:\/\/www.newsbeep.com\/ie\/49158\/"},"modified":"2025-09-28T17:55:18","modified_gmt":"2025-09-28T17:55:18","slug":"ai-hallucinations-what-are-they-and-are-they-always-bad","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/ie\/49158\/","title":{"rendered":"AI Hallucinations: What Are They and Are They Always Bad?"},"content":{"rendered":"

Hallucinations are a frequent point of concern in conversations about AI in healthcare. But what do they actually mean in practice? This was the topic of discussion during a panel held last week at the MedCity INVEST Digital Health Conference<\/a> in Dallas.<\/p>\n

According to Soumi Saha, senior vice president of government affairs at Premier Inc. and moderator of the session, AI hallucinations are when AI \u201cuses its imagination,\u201d which can sometimes hurt patients because it could be providing wrong information.<\/p>\n

One of the panelists \u2014 Jennifer Goldsack, founder and CEO of the Digital Medicine Society \u2014 described AI hallucinations as the \u201ctech equivalent of bullshit.\u201d Randi Seigel, partner at Manatt, Phelps & Phillips, defined it as when AI makes something up, \u201cbut it sounds like it\u2019s a fact, so you don\u2019t want to question it.\u201d Lastly, Gigi Yuen, chief data and AI officer of Cohere Health, said hallucinations are when AI is \u201cnot grounded\u201d and \u201cnot humble.\u201d<\/p>\n

But are hallucinations always bad? Saha posed this question to the panelists, wondering if a hallucination can help people \u201cidentify a potential gap in the data or a gap in the research\u201d that shows the need to do more.<\/p>\n

Yuen said that hallucinations are bad when the user doesn\u2019t know that the AI is hallucinating.<\/p>\n

However, \u201cI will be completely happy to have a brainstorming conversation with my AI chatbot, if it\u2019s willing to share with me how comfortable they are with what they say,\u201d she noted.<\/p>\n

Goldsack equated AI hallucinations to clinical trials data, arguing that missing data can actually tell researchers something. For example, when conducting clinical trials on mental health, missing data can actually be a signal that someone is doing really well because they\u2019re \u201cliving their life\u201d instead of daily recording their symptoms. However, the healthcare industry often uses blaming language when there is missing data, stating that there is a lack of adherence among patients, instead of reflecting on what the missing data actually means.<\/p>\n

She added that the healthcare industry tends to put a lot of \u201cvalue judgments onto technology,\u201d but technology \u201cdoesn\u2019t have a sense of values.\u201d So if the healthcare industry experiences hallucinations with AI, it\u2019s up to humans to be curious about why there\u2019s a hallucination and use critical thinking.<\/p>\n

\u201cIf we can\u2019t make these tools work for us, it\u2019s unclear to me how we actually have a sustainable healthcare system in the future,\u201d Goldsack said. \u201cSo I think we have a responsibility to be curious and to be sort of on the lookout for these sorts of things, and thinking about how we actually compare and contrast with other legal frameworks, at least as a jumping off point.\u201d<\/p>\n

Seigel of Manatt, Phelps & Phillips, meanwhile, stressed the importance of squeezing AI into the curriculum for med and nursing students, including how to understand it and ask questions.<\/p>\n

\u201cIt certainly isn\u2019t going to be sufficient to click through a course in your annual training that you\u2019re spending three hours doing already to tell you how to train on AI. \u2026 I think it has to be iterative, and not just something that\u2019s taught one time and then part of some refresher course that you click through during all the other annual trainings,\u201d she said.<\/p>\n","protected":false},"excerpt":{"rendered":"Hallucinations are a frequent point of concern in conversations about AI in healthcare. But what do they actually…\n","protected":false},"author":2,"featured_media":49159,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[103,397,396,61,60],"class_list":{"0":"post-49158","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-healthcare","8":"tag-health","9":"tag-health-care","10":"tag-healthcare","11":"tag-ie","12":"tag-ireland"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/49158","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=49158"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/49158\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/49159"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=49158"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=49158"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=49158"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}