{"id":250452,"date":"2026-01-21T20:02:09","date_gmt":"2026-01-21T20:02:09","guid":{"rendered":"https:\/\/www.newsbeep.com\/il\/250452\/"},"modified":"2026-01-21T20:02:09","modified_gmt":"2026-01-21T20:02:09","slug":"it-was-notorious-for-getting-things-wrong-now-its-assisting-your-doctor","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/il\/250452\/","title":{"rendered":"It was notorious for getting things wrong. Now it\u2019s assisting your doctor."},"content":{"rendered":"

Sign up for the Slatest<\/a> to get the most insightful analysis, criticism, and advice out there, delivered to your inbox daily.<\/p>\n

Asking a general-use chatbot for health help used to seem like a shot in the dark\u2014just two years ago, a study found that ChatGPT could diagnose only 2 in 10 pediatric cases<\/a> correctly. Among Google Gemini\u2019s early recommendations were eating one small rock a day and using glue to help cheese stick to pizza<\/a>. Last year, a nutritionist ended up hospitalized<\/a> after taking ChatGPT\u2019s advice to replace salt in his diet with sodium bromide.<\/p>\n

Now A.I. companies have begun releasing health-specific chatbots for both consumers and health care professionals. This month, OpenAI announced<\/a> ChatGPT Health, which allows regular people to connect their medical records and health data to A.I. for (theoretically) more accurate responses to their health queries. It also released ChatGPT for Healthcare, a service that is already in use by hospitals across the country. OpenAI isn\u2019t the only one\u2014Anthropic announced its own chatbot, Claude for Healthcare, designed to help doctors with day-to-day tasks like retrieving medical records and to help patients better communicate with their providers.<\/p>\n

So how could these chatbots be an improvement over regular old chatbots? \u201cWhen talking about something designed specifically for health care, it should be trained on health care data,\u201d says Torrey Creed, an associate professor of psychiatry researching A.I. at the University of Pennsylvania. This means that a chatbot shouldn\u2019t have the option to pull from unreliable sources like social media. The second difference, she says, is ensuring that users\u2019 private data isn\u2019t sold or used to train models. Chatbots created for the health care sector are required to be HIPAA compliant. Bots that prompt consumers to directly chat with them about symptoms are designed only to connect the dots, and protecting consumer data is a matter of having robust privacy settings.<\/p>\n

I spoke to Raina Merchant, the executive director of the Center for Health Care Transformation and Innovation at UPenn, about what patients need to know as they navigate the changing A.I. medical landscape, and how doctors are already applying the tech. Merchant says A.I. has a lot of potential\u2014but that, for now, it should be used with caution.<\/p>\n

How is the health care system currently using these chatbots and A.I.?<\/p>\n

It\u2019s a really exciting area. At Penn, we have a program called Chart Hero<\/a>, which can be thought of like a ChatGPT embedded into a patient\u2019s health record. It\u2019s an A.I. agent I can prompt with specific questions to help find information in a chart or make calculations for risk scores or guidance. Since it\u2019s all embedded, I don\u2019t have to go look at separate sources.<\/p>\n

Using it, I can spend more time really talking to patients and have more of that human connection\u2014because I\u2019m spending less time doing chart digging or synthesizing information from different areas. It\u2019s been a real game changer.<\/p>\n

There\u2019s a lot of work in the ambient space, where A.I. can listen after patients have consented and help generate notes. Then there\u2019s also a lot of work in messaging interfaces. We have a portal where patients can send questions at any time using A.I. to help identify ways, still with a human in the loop, to be able to accurately answer information.<\/p>\n

What does having a human in the loop look like?<\/p>\n

Many hospital chatbots are intentionally supervised by humans. What might feel automated is often supported by people behind the scenes. Having a human makes sure that there are some checks and balances.<\/p>\n

So a completely consumer-facing product like ChatGPT Health wouldn\u2019t have a human in the loop. You can just sit on the couch by yourself and have A.I. answer your health questions. What would you recommend that patients use ChatGPT Health for? What are the limitations?<\/p>\n

I think of A.I. chatbots as tools. They are not clinicians. Their goal is to make care easier to access and navigate. They are good at guidance, but not so much judgment. They can help you understand next steps, but I wouldn\u2019t use them for making medical decisions.<\/p>\n

I really like the idea of using it to think through questions to ask your doctor. Going to a medical appointment, people can have certain emotions. Feeling like you\u2019re going in more prepared, that you thought of all the questions, can be good.<\/p>\n

Let\u2019s say I have a low-grade fever. Is it a good idea to ask ChatGPT Health what to do?<\/p>\n

If you are at the point of making a decision, that\u2019s when I would engage a physician. I see real value in using the chatbot as a tool for understanding next steps but not for making a decision.<\/p>\n

So how reliable are these new health chatbots at diagnosing conditions?<\/p>\n

They have a tremendous amount of information that can be informative for both patients and clinicians. What we don\u2019t know yet is when they hallucinate, or when they veer from guidelines or recommendations.<\/p>\n

It won\u2019t be clear when the bot is making something up.<\/p>\n

There\u2019s a couple things that I tell patients: Check for consistency, go to trusted sources to validate information, and trust their instincts. If something sounds too good to be true, have a certain amount of hesitancy making any decisions based on the bot\u2019s information.<\/p>\n

<\/p>\n

\"\"<\/p>\n

\n Alison Block
\n I\u2019m a Doctor. I Never in a Million Years Thought I\u2019d Do What I\u2019m Doing Now to Connect With Patients.
\n Read More\n <\/p>\n

<\/a><\/p>\n

What sources should patients be using to verify A.I.?<\/p>\n

<\/p>\n

Should You Take Advantage of the Latest Controversial Development in Health Care? Your Doctor Already Is.
\n <\/a><\/p>\n

I rely on the big recognizable names, like information from the American Heart Association or other large medical associations that might have guidelines or recommendations. When it gets to the question \u201cShould I trust the chatbot?,\u201d that\u2019s probably when it\u2019s valuable to work with your health care professional.<\/p>\n

Is the data that patients put into health chatbots secure?<\/p>\n

My recommendation for any patient would be to not share personal details, like your name, address, medical record number, or prescription IDs, because it\u2019s not the environment we use for protecting patient information\u2014in the same way that I wouldn\u2019t enter my Social Security number into a random website or Google interface.<\/p>\n

Does this include health care chatbots provided through hospitals or health centers?<\/p>\n

If a hospital is providing a chatbot and [is very clear and transparent] about how the information is being used, and health information is protected, then I would feel comfortable entering my information there. But for something that didn\u2019t have transparency around who owns the data, how it\u2019s used, etc., I would not share my personal details.<\/p>\n

<\/p>\n

Sign up for Slate’s evening newsletter.<\/p>\n","protected":false},"excerpt":{"rendered":"Sign up for the Slatest to get the most insightful analysis, criticism, and advice out there, delivered to…\n","protected":false},"author":2,"featured_media":250453,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[343,163,521,85,46,2409],"class_list":{"0":"post-250452","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-healthcare","8":"tag-artificial-intelligence","9":"tag-health","10":"tag-healthcare","11":"tag-il","12":"tag-israel","13":"tag-medicine"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/250452","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/comments?post=250452"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/posts\/250452\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media\/250453"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/media?parent=250452"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/categories?post=250452"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/il\/wp-json\/wp\/v2\/tags?post=250452"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}