In its current state, industry-driven artificial intelligence (AI) can’t be ethically or responsibly implemented in public health systems, write physicians in the American Journal of Public Health.
In the commentary yesterday, Terry Adirim, MD, MPH, of the Uniformed Services University of Health Sciences, and Amy Molten, MD, of Tufts University School of Medicine, said that, without stronger governance, AI poses risks to patients—especially those from historically marginalized groups such as children, racial minorities, indigenous people, older adults, homeless or incarcerated people, undocumented families, rural residents, and people with disabilities.
Despite the potential for AI to improve surveillance, behavioral interventions, disease prevention, risk communication, and population health, the authors said it is being incorporated haphazardly, without adequate vetting, into public health practice.
“Although AI demonstrates promise for earlier disease detection, improved intervention targeting, and cost-effective augmentation of limited public health capacity, many tools are being deployed amid fragmented regulatory frameworks, validation standards, and equity safeguards that govern other health interventions,” Adirim and Molten wrote. “Consequently, AI can introduce significant ethical, operational, and equity-related risks.”
Misinformation, outdated recommendations
The authors noted that public health systems operate within chronic structural constraints such as data fragmentation, underinvestment, workforce shortages, and health inequities and that data used to train AI models can reflect historical trends of poor healthcare access, structural racism, and social inequality.
Tools such as chatbots used for symptom screening, testing, and vaccine promotion may have been trained on preliminary or outdated evidence and may lack cultural sensitivity and the ability to answer complex questions.
Generative AI, or tools that create brand-new content based on patterns learned from existing data, may intentionally or unintentionally produce false health information that can mislead people into making poor decisions. These risks, Adirim and Molten said, are often downplayed by vendors and the agencies that adopt the technologies.
“These risks are amplified by AI-enabled deepfakes in text, audio, or video form that can impersonate clinicians or public officials and make false claims appear authoritative, further eroding trust in reliable sources and weakening the environment of trust on which effective public health depends,” they wrote.
AI, the authors said, can generate biased conclusions when analyzing diverse datasets and risk manipulation and misinformation while threatening autonomy. What’s more, AI tools using commercial data such as location tracking raise questions about consent, public health surveillance, and consumer privacy.
Prioritization of commercial interests over health, equality
Apps driven by AI also may, under a facade of offering tailored, evidence-based guidance, promote uniform health recommendations such as annual mammograms for all women aged 40 and older, despite ongoing debate in the medical community about optimal screening intervals and consideration of the balance between saving lives and the harms of false-positive test results and overdiagnosis.
Responsible use of AI in public health practice must therefore be calibrated to this diversity and combine cross-cutting governance with population-specific validation, community engagement, and safeguards tailored to each group.
“This dynamic is especially harmful for populations with limited health literacy, English proficiency, or financial resources, who may be least able to critically evaluate algorithmic prompts or absorb the downstream costs of unnecessary follow-up,” the authors wrote.
Current approaches to AI integration into public health often prioritize technical capabilities, commercial interests, and institutional efficiency rather than equality, accountability, and the reduction of harms to underserved communities. The lack of active identification and reduction of risks such as misclassification, stigmatization, and exclusion can harm vulnerable groups.
“The threats to children’s developmental privacy differ from the data sovereignty concerns of Indigenous communities; the misclassification risks for older adults differ from the surveillance-related fears of undocumented families or incarcerated people; the digital divide barriers facing rural and global populations differ from the engagement-based manipulation risks faced by people with behavioral health conditions,” they wrote.
“Responsible use of AI in public health practice must therefore be calibrated to this diversity and combine cross-cutting governance with population-specific validation, community engagement, and safeguards tailored to each group,” they added.
Third-party certification needed
Adirim and Molten also presented strategies to assess AI tools, such as:
Mandatory equity-impact evaluationsValidations in the settings for which they are intendedAlignment with data-sovereignty principlesTransparency regarding when and how AI is used in communitiesContinued investment in the public health workforce capacity needed to test the toolsHuman oversight of important decisionsNational standards for AI transparency and equity testing
Rather than relying on spotty or absent industry self-regulation, a coherent governance architecture, national standards, and a third-party certification process similar to those for aviation and medical devices should be put in place, the authors said.
“Public health leaders must help shape AI design and deployment in ways that strengthen trust, dismantle conditions that harm communities, and advance population health,” they concluded.