{"id":10277,"date":"2025-07-21T06:59:29","date_gmt":"2025-07-21T06:59:29","guid":{"rendered":"https:\/\/www.newsbeep.com\/au\/10277\/"},"modified":"2025-07-21T06:59:29","modified_gmt":"2025-07-21T06:59:29","slug":"toward-the-autonomous-ai-doctor-quantitative-benchmarking-of-an-autonomous-agentic-ai-versus-board-certified-clinicians-in-a-real-world-setting","status":"publish","type":"post","link":"https:\/\/www.newsbeep.com\/au\/10277\/","title":{"rendered":"Toward the Autonomous AI Doctor: Quantitative Benchmarking of an Autonomous Agentic AI Versus Board-Certified Clinicians in a Real World Setting"},"content":{"rendered":"

Abstract<\/p>\n

Background Globally we face a projected shortage of 11 million healthcare practitioners by 2030, and administrative burden consumes 50% of clinical time. Artificial intelligence (AI) has the potential to help alleviate these problems. However, no end-to-end autonomous large language model (LLM)-based AI system has been rigorously evaluated in real-world clinical practice. In this study, we evaluated whether a multi-agent LLM-based AI framework can function autonomously as an AI doctor in a virtual urgent care setting.<\/p>\n

Methods We retrospectively compared the performance of the multi-agent AI system Doctronic and board-certified clinicians across 500 consecutive urgent-care telehealth encounters. The primary end points: diagnostic concordance, treatment plan consistency, and safety metrics, were assessed by blinded LLM-based adjudication and expert human review.<\/p>\n

Results The top diagnosis of Doctronic and clinician matched in 81% of cases, and the treatment plan aligned in 99.2% of cases. No clinical hallucinations occurred (e.g., diagnosis or treatment not supported by clinical findings). In an expert review of discordant cases, AI performance was superior in 36.1%, and human performance was superior in 9.3%; the diagnoses were equivalent in the remaining cases.<\/p>\n

Conclusions In this first large-scale validation of an autonomous AI doctor, we demonstrated strong diagnostic and treatment plan concordance with human clinicians. These findings indicate that multi-agent AI systems can achieve comparable clinical decision-making to human providers and offer a potential solution to healthcare workforce shortages.<\/p>\n

\"Figure\"<\/a>Competing Interest Statement<\/p>\n

All authors are equity owners of Doctronic Inc.<\/p>\n

Funding Statement<\/p>\n

This study did not receive any funding<\/p>\n

Author Declarations<\/p>\n

I confirm all relevant ethical guidelines have been followed, and any necessary IRB and\/or ethics committee approvals have been obtained.<\/p>\n

Yes<\/p>\n

The details of the IRB\/oversight body that provided approval or exemption for the research described are given below:<\/p>\n

Ethics committee of Doctronic Research Group gave ethical approval for this work. The work was deemed to utilize only de-identified and unlinked data from an EHR for the study.<\/p>\n

I confirm that all necessary patient\/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient\/participant\/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.<\/p>\n

Yes<\/p>\n

I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).<\/p>\n

Yes<\/p>\n

I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.<\/p>\n

Yes<\/p>\n

Data Availability<\/p>\n

All data produced in the present study are available upon reasonable request to the authors<\/p>\n","protected":false},"excerpt":{"rendered":"Abstract Background Globally we face a projected shortage of 11 million healthcare practitioners by 2030, and administrative burden…\n","protected":false},"author":2,"featured_media":10278,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[34],"tags":[64,63,137,500],"class_list":["post-10277","post","type-post","status-publish","format-standard","has-post-thumbnail","category-healthcare","tag-au","tag-australia","tag-health","tag-healthcare"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/10277","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/comments?post=10277"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/posts\/10277\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media\/10278"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/media?parent=10277"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/categories?post=10277"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/au\/wp-json\/wp\/v2\/tags?post=10277"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}