Dr. Nicholas Gavin, an emergency medicine doctor at Mount Sinai in New York City, was working an overnight shift last summer when a patient came in with a puzzling set of symptoms. Within seconds, his three younger colleagues \u2014 two medical students and a resident \u2014 were consulting a free artificial-intelligence-powered app for physicians, OpenEvidence.<\/p>\n
Dr. Gavin soon learned that they were far from outliers. A third of Mount Sinai\u2019s 9,000 doctors were already regular OpenEvidence users, the health system\u2019s executives found out in a meeting last year with the start-up\u2019s leaders.<\/p>\n
\u201cThat was an \u2018aha\u2019 moment for our leadership,\u201d said Dr. Gavin, who is also the system\u2019s chief clinical innovation officer.<\/p>\n
OpenEvidence\u2019s A.I. app, essentially a chatbot for medicine, has become a viral hit with physicians. Talk to a doctor and chances are he or she uses the app to ask specific medical questions or bounce ideas off it in a diagnostic dialogue.<\/p>\n
More than half of the nation\u2019s physicians are regular users. Last month, they used it for 30 million questions and consultations, nearly twice the volume from six months earlier, according to the start-up. A separate survey last year of 1,000 physicians<\/a> found that 45 percent of them used the app, nearly triple the percentage who used ChatGPT, according to Offcall, a career information service for doctors.<\/p>\n That growth propelled the start-up to a $12 billion valuation in January, up from $3.5 billion last July.<\/p>\n But doctors\u2019 quick adoption of the app since its introduction in 2024 \u2014 one of a handful of A.I.-enhanced programs on the market seeking to win over physicians \u2014 has heightened concerns about how and when the technology should be used in life-or-death situations. In a high-stakes field like medicine, health care systems are navigating thorny matters of patient privacy, safety and trust, as well as the limitations of the technology itself.<\/p>\n \u201cIt\u2019s not an oracle, it\u2019s a tool,\u201d said Daniel Nadler, founder and chief executive of OpenEvidence. \u201cKnowledge and knowledge workers still matter.\u201d<\/p>\n The doctor\u2019s office has been a target for computer-assisted decision making for decades, with very limited success until the recent advances of A.I.<\/p>\n The first wave of A.I. in medicine focused on easing the heavy burden of documentation that contributes to physician burnout with transcriptions and summaries of patient visits, called A.I. scribe software. The second wave, which is just getting underway, aims to use A.I. to assist doctors with reliable information and advice to guide diagnosis and treatment while at a patient\u2019s bedside.<\/p>\n The competition has intensified in recent months. UpToDate<\/a>, a popular legacy electronic reference for doctors, has given its service an A.I. makeover with a chatbot interface. Doximity,<\/a> an online professional network for physicians, bought an A.I. start-up that mines medical literature and generates summaries. Abridge<\/a>, a fast-growing A.I. scribe maker, is adding decision-support tools. And last month, OpenAI introduced ChatGPT for Clinicians<\/a>.<\/p>\n OpenEvidence became a front-runner in part because it exclusively used medical journals and other high-quality research as data to train its A.I. models. Physicians can ask the app specific questions or enter the characteristics and symptoms of a patient and ask for potential explanations. The app is compliant with the federal law that protects patient health information, and physicians are told not to enter any personally identifying information.<\/p>\n OpenEvidence responds with a summary of most likely diagnoses, and then offers other \u201cmost important not to miss diagnoses.\u201d Each has links to the research articles that inform the summaries.<\/p>\n \u201cA.I. is solving some of the problems that have long plagued the practice of medicine,\u201d said Dr. Raja-Elie Abdulnour, chief clinical innovation officer at NEJM Group, which publishes The New England Journal of Medicine. \u201cThese tools just didn\u2019t exist before, and that\u2019s why people are so excited about them now.\u201d<\/p>\n Yet the early enthusiasm should be tempered with a large dose of caution, medical experts agree. The research so far<\/a> into the benefits and shortcomings of A.I. in medicine is decidedly mixed.<\/p>\n A.I. has aced standard licensing exams and outperformed human doctors in diagnosing certain cases. But A.I. has also stumbled, failing to accurately summarize research papers or giving wrong answers to diagnostic questions. And it isn\u2019t going to replace humans anytime soon.<\/p>\n \u201cThe potential for A.I. is great, but we\u2019re not there yet,\u201d said Dr. Eric Topol, a cardiologist and an executive vice president at Scripps Research in San Diego. \u201cIt hasn\u2019t really been tested and demonstrated in the messy, real world of medicine.\u201d<\/p>\n Dr. Topol is a co-author of a recent paper, \u201cThe Illusion of Readiness in Health A.I.,\u201d<\/a> which found \u201csignificant competency gaps\u201d in the capability of big A.I. systems when applied to health care.<\/p>\n The evaluations so far have largely focused on the performance of the so-called large language models of big tech companies like OpenAI and Google, which are trained on data across the open internet.<\/p>\n