Sign up to see the future, today<\/p>\n
Can\u2019t-miss innovations from the bleeding edge of science and tech<\/p>\n
Just weeks after the largest nurses strike<\/a> in the New York City history, the CEO of NYC Health and Hospitals has a bold vision for a future where AI, not human radiologists, examines and diagnoses X-rays.<\/p>\n At a panel held by Crain\u2019s New York Business, Mitchell Katz, president and CEO of New York\u2019s 11-hospital public benefit corporation, made overt gestures at his desire to replace highly trained radiology experts with visual language AI models, Radiology Business reported<\/a>.<\/p>\n \u201cWe could replace a great deal of radiologists with AI at this moment, if we are ready to do the regulatory challenge,\u201d Katz said at the panel.<\/p>\n One example he gave, according to Radiology, would affect women\u2019s healthcare in particular, by automating breast cancer screening with AI tools. By sidelining radiologists until an AI system flags a reading as abnormal, Katz declared, hospitals could achieve \u201cmajor savings.\u201d<\/p>\n Mohammed Suhail, a radiologist at North Coast Imaging in San Diego, told Radiology that Katz\u2019s comments are \u201cundeniable proof that confidently uninformed hospital administrators are a danger to patients,\u201d and are \u201ceasily duped by AI companies that are nowhere near capable of providing patient care.\u201d<\/p>\n \u201cAny attempt to implement AI-only reads would immediately result in patient harm and death,\u00a0and only someone with zero understanding of radiology would say something so naive,\u201d Suhail continued. \u201cBut in some sense, they\u2019re correct: hospitals are happy to cut costs even if it means patient harm, as long as it\u2019s legal.\u201d<\/p>\n Indeed, a growing body of research suggests that AI in the X-ray room is a disaster waiting to happen.<\/p>\n In a yet-to-be-peer-reviewed study<\/a>, Stanford researchers found that AI chest X-ray tools built on frontier AI models<\/a> can ace medical benchmark tests without ever seeing actual images of X-rays. Rather than admit that the images are missing, the highest scoring AI systems would engage in what amounts to a cheap parlor trick: constructing an elaborate explanation for findings on X-rays it never had access to in the first place.<\/p>\n This situation goes a step above mere AI hallucinations into what the researchers call an AI \u201cmirage.\u201d Unlike the generative AI errors<\/a> we\u2019ve come to expect, the AI mirage is incredibly rational from start to finish. The issue is that these mirages aren\u2019t based on anything, meaning usual hallucination safeguards<\/a> aren\u2019t enough to deter them.<\/p>\n \u201cIn this epistemic mimicry, the model simulates the entire perceptual process that would have led to the answer,\u201d the Stanford scientists wrote. \u201cThis helps explain why reasoning traces, on their own, cannot certify visual reasoning: the trace may be fluent, coherent, and apparently image-based while being anchored to no image at all.\u201d<\/p>\n On top of reinforcing previous research suggesting visual language AI models are functionally blind<\/a>, this study has major implications for any hospital turning to AI to trim its radiology unit \u2014 not to mention any patient unlucky enough to be on the receiving end of a medical imaging mirage.<\/p>\n