{"id":773783,"date":"2026-09-17T16:32:09","date_gmt":"2026-09-17T16:32:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/773783\/"},"modified":"2026-09-17T16:32:09","modified_gmt":"2026-09-17T16:32:09","slug":"chatgpt-health-solution-or-experiment","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/773783\/","title":{"rendered":"ChatGPT Health: Solution or Experiment?"},"content":{"rendered":"

More than 300 million people consult ChatGPT for health-related questions every week<\/a>. To accommodate the demand, OpenAI quietly rolled out ChatGPT Health last month to a select group of US adults; a soft opening of sorts. For the world\u2019s first patient-focused AI, there was surprisingly little fanfare, and I can\u2019t help but think there was a reason.<\/p>\n

ChatGPT Health is based on the same underlying architecture as regular ChatGPT, but with a key difference: with permission, the AI can link to your medical records and download health metrics recorded by Apple wearables, such as the Apple Watch. Though initially a separate tab in ChatGPT\u2019s interface, it\u2019s now seamlessly integrated; an operational decision by OpenAI after seeing users posing health questions to the chatbot organically during daily conversations. An OpenAI press release<\/a> said ChatGPT Health can \u201chelp explain a visit note, understand lab results, dig into doctor-patient discussions, and help prepare questions for follow-up appointments\u2026\u201d The platform can optimize the user\u2019s experience with more \u201cpersonalized conversations.\u201d<\/p>\n

But this initiative has tension at its heart. On the one hand, OpenAI qualifies the rollout by saying it\u2019s not for diagnosis or treatment and is \u201cnot designed to replace the care and judgment of qualified medical professionals.\u201d On the other hand, that\u2019s exactly what the public will use it for, and OpenAI knows it. “The reality is, we’re living in a time when not everyone has access to quality care or care in a timely manner,\u201d said one executive<\/a> to reporters last month. \u201cThe average doctor appointment in the United States is less than 15 minutes… You’re often left on your own to sort through a lot of fragmented information.\u201d For many, the black box that is ChatGPT Health will be used in the absence of a physician; after all, if users had access to their doctor, what would they need from AI?<\/p>\n

A Limited Skillset<\/p>\n

When ChatGPT launched in 2022, it operated exclusively on its \u201cparametric\u201d architecture; that is, it was entirely constrained by the reams of text-based data on which it had been trained, hence the name Large Language Model (LLM). A query such as \u201cWhat\u2019s the best source of vitamin C?\u201d would prompt the AI to draw on the statistical model it developed during training to predict the most likely word sequence in a response. The model\u2019s strength was its conversation mimicry: fluent, confident, and generally articulate. The weakness was that the training data were scattered, comprising everything from open-access research papers and mainstream articles to blog posts, Q&A forums (e.g., Reddit), and even social media<\/a>\u2014a cesspit of misinformation. A user asking about vitamin C would need to hope that \u201cguava, kiwi, and citrus fruit\u201d appeared more often in the training data than \u201csausages and caviar.\u201d Worse still, if the query wasn\u2019t covered in training, the AI would fabricate an answer, like a teenager<\/a> trying to impress his friends. The hallucination was born.<\/p>\n

But AI is evolving at a pace that makes our own evolutionary history appear positively glacial. In the spring of 2023, ChatGPT was upgraded with its RAG plugin, short for retrieval-augmented generation. This software bolt-on lets current AI generations perform online searches in real-time when the AI deems its training data insufficient for a full-throated reply. And while this hands AI the keys to the full wealth of human digital knowledge, there\u2019s a wrinkle: finding data isn\u2019t the same as evaluating it. You can give someone access to the entire PubMed catalog, but it doesn\u2019t mean they\u2019ll be able to make clinical decisions. So, for AI and humans alike, access to information isn\u2019t the limiting factor; rather, it\u2019s the ability to distinguish among quality sources, weigh evidence, and apply knowledge and experience. These are qualities AI doesn\u2019t possess.<\/p>\n

The Shallow Body of Evidence<\/p>\n

Programming an AI with these high-level skills is certainly possible, but it\u2019s proving complex. And published audits on health-related content suggest better AI reasoning will be crucial to the success of initiatives like ChatGPT Health.<\/p>\n

One recent study<\/a> found that chatbots given medical queries produced \u201cproblematic\u201d responses one-fifth to one-half of the time, and \u201cunsafe\u201d responses varied from 5 to 13% of the total. Our own audit<\/a> of five AI chatbots, published earlier this year in the British Medical Journal, recorded \u201cproblematic\u201d answers in roughly 50% of chatbot responses to health questions in misinformation-prone fields; one-fifth were expert-rated as \u201chighly problematic,\u201d and could potentially cause harm if followed. These are two among dozens of studies on the efficacy of AI health-related outputs, showing them to be inconsistent and unreliable.<\/p>\n

The limited data on ChatGPT Health, in particular, is no less disconcerting. In a structured test of triage recommendations, Ramaswamy and colleagues at Mount Sinai Health System, NY, reported<\/a> that 52% of gold-standard emergencies were under-triaged, directing patients with diabetic ketoacidosis or impending respiratory failure to 24\u201348\u2009h evaluation rather than the emergency department. A preprint<\/a> awaiting peer review similarly indicates that ChatGPT Health agrees with nurse and physician triage decisions only 50% of the time.<\/p>\n

A 50% under-triage rate for genuine emergencies is not a marginal performance defect; it has major implications with asymmetric consequences: over-triage a minor ailment risks stressing a medical system already near breaking point, but under-triaging a real emergency can be catastrophic. The evidence suggests AI is not a suitable tool for first-contact triage. Try telling that to the >40 million people who turn to ChatGPT with healthcare questions every day<\/a>.<\/p>\n

***<\/p>\n

None of this means that lower-risk functions are unsafe. For example, the AI may accurately translate clinical terminology, organize medical records, plot and analyze lab results chronologically, and help users prepare questions for their clinician. The difficulty is that ChatGPT Health does not maintain a dependable boundary<\/a> between those functions and clinical judgment; the moment it interprets symptoms and offers advice, it becomes a functioning triage system, irrespective of OpenAI\u2019s disclaimer. There\u2019s a mismatch between what the system is designed to do and how the end user will ultimately use it. And there\u2019s no evidence that the AI itself understands that limitation.<\/p>\n

The good news is that OpenAI continues to evaluate ChatGPT Health using an assessment matrix it developed called Health Bench<\/a>. Created in collaboration<\/a> with 262 physicians from 60 countries, across 26 medical specialties, Health Bench periodically audits thousands of conversations to identify and repair weaknesses in the platform. It\u2019s a robust and necessary step, but no data have been published yet, and it\u2019s clearly a work in progress rather than a finished product.<\/p>\n

Such public-facing initiatives continue even as industry leaders call for greater restraint. In September 2026, OpenAI\u2019s Sam Altman joined<\/a> Anthropic\u2019s Dario Amodei and Elon Musk in calling for a slowdown in advanced AI development, with stronger safeguards and independent oversight. The same principle applies to healthcare: safety checks must keep pace with capability. That isn\u2019t happening.<\/p>\n

Consider the consequences if Ford rolled out a new truck before proving it was safe to drive. We wouldn\u2019t accept a half-baked product or the manufacturer\u2019s reassurance that it was a \u201cwork in progress.\u201d We should demand the same accountability from companies with so much potential influence on public health. Without that accountability, we are the last phase of the experiment.<\/p>\n","protected":false},"excerpt":{"rendered":"More than 300 million people consult ChatGPT for health-related questions every week. To accommodate the demand, OpenAI quietly…\n","protected":false},"author":2,"featured_media":773784,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[59,57,58,50,56,54,55],"class_list":["post-773783","post","type-post","status-publish","format-standard","has-post-thumbnail","category-united-kingdom","tag-gb","tag-great-britain","tag-greatbritain","tag-news","tag-uk","tag-united-kingdom","tag-unitedkingdom"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/773783","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=773783"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/773783\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/773784"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=773783"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=773783"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=773783"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}