{"id":390107,"date":"2026-01-25T18:51:08","date_gmt":"2026-01-25T18:51:08","guid":{"rendered":"https:\/\/www.newsbeep.com\/uk\/390107\/"},"modified":"2026-01-25T18:51:08","modified_gmt":"2026-01-25T18:51:08","slug":"we-must-not-let-ai-pull-the-doctor-out-of-the-visit-for-low-income-patients-leah-goodridge-and-oni-blackstock","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/uk\/390107\/","title":{"rendered":"We must not let AI \u2018pull the doctor out of the visit\u2019 for low-income patients | Leah Goodridge and Oni Blackstock"},"content":{"rendered":"

In southern California, where rates of homelessness are among the highest in the nation, a private company, Akido Labs<\/a>, is running clinics for unhoused patients and others with low incomes. The caveat? The patients are seen by medical assistants who use artificial intelligence (AI) to listen to the conversations, then spit out potential diagnoses and treatment plans, which are then reviewed by a doctor. The company\u2019s goal, its chief technology officer told the MIT Technology Review, is to \u201cpull the doctor out of the visit<\/a>\u201d.<\/p>\n

This is dangerous. Yet it\u2019s part of a larger trend where generative AI is being pushed into healthcare for medical professionals. In 2025, a survey<\/a> by the American Medical Association reported that two out of three physicians used AI to assist with their daily work, including diagnosing patients. One AI startup raised $200m<\/a> to provide medical professionals with an app dubbed \u201cChatGPT for doctors\u201d. US lawmakers are considering a bill<\/a> that would recognize AI as able to prescribe medication. While this trend of AI in healthcare affects almost all patients, it has a deeper impact on people with low incomes who already face substantial barriers to care and higher rates of mistreatment in healthcare settings. People who are unhoused and have low incomes should not be testing grounds for AI in healthcare. Instead, their voices and priorities should drive if, how, and when AI is implemented in their care.<\/p>\n

The rise of AI in healthcare didn\u2019t happen in a vacuum. Overcrowded hospitals, overworked clinicians and relentless pressure for medical offices to run seamlessly, shuttling patients in and out of a large for-profit healthcare system, set the conditions. The demands on healthcare workers are often compounded in economically disadvantaged communities where healthcare settings are often under-resourced and patients are uninsured, with a greater burden of chronic health conditions due to racism and poverty.<\/p>\n

Here is where someone might ask, \u201cIsn\u2019t something better than nothing?\u201d Well, actually, no. Studies show that AI-enabled tools generate inaccurate diagnoses. A 2021 study<\/a> in Nature Medicine examined AI algorithms trained on large, chest X-ray datasets for medical imaging research and found that these algorithms systematically under-diagnosed Black and Latinx patients, patients recorded as female and patients with Medicaid insurance. This systematic bias risks deepening health inequities for patients already facing barriers to care. Another study<\/a>, published in 2024, found that AI misdiagnosed breast cancer screenings among Black patients \u2013 the odds of false positives for Black patients screened for breast cancer was greater than for their white counterparts. Due to algorithmic bias, some clinical AI tools have notoriously performed worse on Black patients and other people of color. That\u2019s because AI isn\u2019t independently \u201cthinking\u201d; it relies on probabilities and pattern recognition, which can reinforce bias for already marginalized patients.<\/p>\n

Some patients aren\u2019t even informed that their health provider or healthcare system is using AI. A medical assistant told the MIT Technology review<\/a> that his patients know an AI system is listening, but he does not tell them that it makes diagnostic recommendations. This harkens back to an era of exploitative medical racism where Black people were experimented on without informed consent and often against their will. Can AI help health providers by speedily giving them information that may allow them to move on to the next patient? Possibly. But the problem is that it might come at the expense of diagnostic accuracy and worsening health inequities.<\/p>\n

And the potential impact goes beyond diagnostic accuracy. TechTonic Justice<\/a>, an advocacy group working to protect economically marginalized communities from the harms of AI, published a groundbreaking report that estimates 92 million Americans<\/a> with low incomes \u201chave some basic aspect of their lives decided by AI\u201d. Those decisions range from how much they receive from Medicaid to whether they are eligible for Social Security administration\u2019s disability insurance<\/a>.<\/p>\n

A real-life example of this is playing out in federal courts right now. In 2023, a group of Medicare Advantage customers sued<\/a> UnitedHealthcare in Minnesota, alleging they were denied coverage because the company\u2019s AI system, nH Predict, mistakenly deemed them ineligible. Some of the plaintiffs are the estates of Medicare Advantage customers; these patients allegedly died as a result of the denial of medically necessary care. UnitedHealth sought to dismiss the case, but in 2025, a judge ruled that the plaintiffs can move forward with some of the claims. A similar case was filed<\/a> in federal court in Kentucky against Humana. There, Medicare Advantage customers alleged that Humana\u2019s use of nH Predict \u201cspits out generic recommendations based on incomplete and inadequate medical records\u201d. That case is also ongoing, with a judge ruling that the plaintiffs\u2019 legal arguments are enough to move forward, surviving the insurance company\u2019s motion to dismiss. While the final decision for these two cases remains pending, they indicate a growing trend of AI being used to decide the health coverage of people with low incomes \u2013 and its pitfalls. If you have financial resources, you can get quality healthcare. But if you are unhoused or have a low income, AI may bar you from even accessing the healthcare entirely. That\u2019s medical classism.<\/p>\n

We should not experiment on patients who are unhoused or have low incomes for an AI rollout. The documented harms are greater than the potential, unproven benefits promised by start-ups and other tech ventures. Given the barriers that people who are unhoused and have low incomes face, it is crucial they receive patient-centered care with a human healthcare provider who listens to their health-related needs and priorities. We cannot create a standard where we rely on a health system in which health practitioners take a backseat while AI \u2013 run by private companies \u2013 takes the lead. An AI system that \u201clistens\u201d in and is developed without rigorous evaluation by the communities themselves disempowers patients by removing their decision-making authority to determine what technologies, including AI, are implemented in their health care.<\/p>\n

Leah Goodridge is a lawyer who worked in homeless prevention litigation for 12 years<\/p>\n

Oni Blackstock, MD, MHS, is a physician, founder and executive director of health justice, and a Public Voices Fellow on technology in the public interest with The OpEd Project<\/p>\n","protected":false},"excerpt":{"rendered":"In southern California, where rates of homelessness are among the highest in the nation, a private company, Akido…\n","protected":false},"author":2,"featured_media":390108,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[20],"tags":[554,733,4308,86,56,54,55],"class_list":{"0":"post-390107","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-artificial-intelligence","8":"tag-ai","9":"tag-artificial-intelligence","10":"tag-artificialintelligence","11":"tag-technology","12":"tag-uk","13":"tag-united-kingdom","14":"tag-unitedkingdom"},"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/390107","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/comments?post=390107"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/posts\/390107\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media\/390108"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/media?parent=390107"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/categories?post=390107"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/uk\/wp-json\/wp\/v2\/tags?post=390107"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}