![\"\"](\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/07\/BUZRZ2STA5H23AN6UJM6O7VQBY.gif\")
<\/p>\nPatients have been warned against turning to artificial intelligence for at-home medical diagnosis after some AI systems failed to recognise crucial symptoms.<\/p>\n
Researchers at Carnegie Mellon University’s School of Computer Science in the US investigated how large language models such as GPT-5, Gemini and Claude responded to basic medical questions aimed at diagnosing illness.<\/p>\n
The analysis showed the large language models (LLMs) occasionally invented false information when asked about details a doctor would routinely refer to when assessing a patient. <\/p>\n
As part of the study, Siddharth Vohra, a robotics institute master\u2019s student, asked popular LLMs to describe a medical image that had been intentionally omitted from the query. <\/p>\n
But rather than requesting the missing image, in 18 per cent of the cases, the models fabricated a diagnosis based on the user\u2019s age, gender and race. <\/p>\n
Misleading <\/p>\n
The results suggested almost one in five diagnoses presented by AI could be false or misleading.<\/p>\n
Computer programmes also made assumptions that some patients had cancer, without referring to a telltale image that could help rule out the disease.<\/p>\n
\u201cA 65-year-old white man asking Claude about a skin mole receives melanoma in nearly every response,\u201d said Mr Vohra, the study’s author. <\/p>\n
\u201cWhen chest X-ray questions are presented, OpenAI\u2019s GPT-5 names sarcoidosis for roughly 77 per cent of young black patients.\u201d<\/p>\n
Cancer specialists estimate that fewer than one in 10,000 moles become cancerous. Sarcoidosis \u2013 an inflammatory disease usually found in the lungs \u2013 is rare and appears in 10 to 60 of every 100,000 people, with some regional variations.<\/p>\n
Mr Vohra said a diagnosis can change when only race, age, or gender is written into the prompt.<\/p>\n
The research highlights flaws in the methods of self-diagnosis in the age of Dr Google and advanced AI.<\/p>\n
By inputting symptoms, patients are bypassing professional medical opinion and placing their health in the hands of a computer programme instead.<\/p>\n
While AI responses can look convincing and authoritative, they are not grounded in fact, and just small changes in the words used can drastically influence a model\u2019s conclusions.<\/p>\n
During the study, Mr Vohra focused on chest X-rays, brain MRI scans and dermatology images in 12 simulated patient profiles. <\/p>\n
The AI models delivered 11,700 responses across Claude, GPT-5 and Google\u2019s Gemini. The findings reinforced a major concern in the AI industry that users often assume models understand more than they actually do.<\/p>\n
\u201cAbout 82 per cent of the time, the models refused to provide a response because no image was attached,\u201d Mr Vohra said. \u201cHowever, in the remaining 18 per cent, the models invented a diagnosis instead of asking for the missing image.\u201d<\/p>\n
\u201cThere\u2019s a general perception that AI models are smart because they perform well on certain tasks. But being good at one thing doesn\u2019t mean they\u2019re good at everything.\u201d <\/p>\n
Convincing diagnosis <\/p>\n
According to the 2026 Edelman Trust Barometer Global Report, out of 1,000 people asked in the UAE, 59 per cent used AI to manage their health. The report also found that seven in 10 people felt confident in their ability to find answers to health questions and to make informed decisions.<\/p>\n
<\/p>\n
\u201cLarge language models reached hospitals before the safeguards did,\u201d said Ahmed Ashoor, chief technology officer for Outworks, a UAE technology service provider.<\/p>\n
\u201cA model produces a fluent answer whether or not the answer is true, and in medicine a confident wrong answer does more damage than no answer at all.<\/p>\n
\u201cDevelopers train most models on western populations, so accuracy drops for regional patients, from genetic risk factors to symptoms a patient describes in Arabic.\u201d<\/p>\n
Data privacy is a further issue holding back the effectiveness of LLMs in healthcare, Mr Ashoor said. \u201cHealth records rank among the most sensitive data a nation holds, and no ministry can hand them to systems outside its control,\u201d he said.<\/p>\n
\u201cAccountability closes the list. When an algorithm shapes a clinical decision, someone must own the outcome, and many health systems cannot yet say who that is.\u201d<\/p>\n
Hospitals <\/p>\n
Radiology is proving to be the most effective area of healthcare where AI is deployed.<\/p>\n
Models flag abnormalities in scans with an accuracy close to that of a specialist, which gives every image a second read without adding a minute to the radiologist’s day. <\/p>\n
Hospital systems based on AI that predicts admissions, manages bed capacity and flags a deteriorating patient hours earlier also save lives without ever writing a prescription.<\/p>\n
Nalla Karunanithy, chief executive of Digital Health and Omnichannel at Aster DM Healthcare, said the group\u2019s hospitals use AI algorithms to help radiologists prioritise urgent cases and detect subtle findings faster.<\/p>\n
Chatbots, however, were not part of hospital operations but were helpful in summarising medical literature, he said.<\/p>\n
\u201cWhere large language models genuinely add value today is around the clinical encounter rather than inside the diagnostic decision itself,\u201d said Mr Karunanithy.<\/p>\n
\u201cGeneral models can carry biases from their training data, including around ethnicity and race, and can present incorrect answers with complete confidence. <\/p>\n
\u201cIn a clinical setting, that confidence without accountability is not something that is encouraged at this point. <\/p>\n
\u201cAny AI that touches a diagnostic pathway at Aster works under physician oversight, and the physician remains responsible for the decision.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"Patients have been warned against turning to artificial intelligence for at-home medical diagnosis after some AI systems failed…\n","protected":false},"author":2,"featured_media":571996,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[218,103,397,4892,396,61,60,1284,329,80,4969],"class_list":["post-571995","post","type-post","status-publish","format-standard","has-post-thumbnail","category-healthcare","tag-artificial-intelligence","tag-health","tag-health-care","tag-health-news","tag-healthcare","tag-ie","tag-ireland","tag-standard","tag-story","tag-technology","tag-uae-desk-team"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/571995","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=571995"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/571995\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/571996"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=571995"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=571995"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=571995"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}