![\"\"](\"https:\/\/www.newsbeep.com\/uk\/wp-content\/uploads\/2026\/01\/SEI_279444031.jpg\")
<\/p>\n<\/p>\n
Many women are using AI for health information, but the answers aren\u2019t always up to scratch<\/p>\n
Oscar Wong\/Getty Images<\/p>\n<\/p>\n
Commonly used AI models fail to accurately diagnose or offer advice for many queries relating to women\u2019s health that require urgent attention.<\/p>\n
Thirteen large language models<\/a>, produced by the likes of OpenAI, Google, Anthropic, Mistral AI and xAI, were given 345 medical queries across five specialities, including emergency medicine, gynaecology and neurology. The queries were written by 17 women\u2019s health researchers, pharmacists and clinicians from the US and Europe.<\/p>\n The answers were reviewed by the same experts. Any questions that the models failed at were collated into a benchmarking test of AI models\u2019 medical expertise that included 96 queries.<\/p>\n Across all the models, some 60 per cent of questions were answered in a way that the human experts had previously said wasn\u2019t sufficient for medical advice. GPT-5 was the best-performing model, failing on 47 per cent of queries, while Ministral 8B had the highest failure rate of 73 per cent.<\/p>\n \u201cI saw more and more women in my own circle turning to AI tools for health questions and decision support,\u201d says team member Victoria-Elisabeth Gruber<\/a> at Lumos AI, a firm that helps companies evaluate and improve their own AI models. She and her colleagues recognised the risks of relying on a technology that inherits and amplifies existing gender gaps in medical knowledge. \u201cThat is what motivated us to build a first benchmark in this field,\u201d she says.<\/p>\n The rate of failure surprised Gruber. \u201cWe expected some gaps, but what stood out was the degree of variation across models,\u201d she says.<\/p>\n The findings are unsurprising because of the way AI models are trained, based in human-generated historical data that has built-in biases, says Cara Tannenbaum<\/a> at the University of Montreal, Canada. They point to \u201ca clear need for online health sources, as well as healthcare professional societies, to update their web content with more explicit sex and gender-related evidence-based information that AI can use to more accurately support women\u2019s health\u201d, she says.<\/p>\n