AI models used to help diagnose medical conditions have a problem: They\u2019re ready and willing to identify patients whose data was used to train them.<\/p>\n
German researchers reported in a Nature paper published<\/a> Wednesday that discriminative AI models – those used to classify data and make predictions about new inputs based on their training sets – are particularly susceptible to membership inference attacks (MIAs) that query the models in an attempt to figure out whether a particular datapoint is included in their training sets.\u00a0<\/p>\n What that means for medical AI models is that any patient whose data is used to educate the bot could be exposed, leading to details about their medical history and diagnoses being leaked. In an analysis of seven medical AI datasets consisting of images, ECG records, and general electronic health records, the team determined that individual patients targeted by such attacks can be identified with \u201cnear-perfect attack success,\u201d which they explain flies in the face of how such models are evaluated for safety.\u00a0<\/p>\n \u201cThe fact that MIAs can achieve near-perfect success rates for individual patients is not adequately captured by the standard evaluation protocol, which measures attack success in aggregate across records,\u201d the researchers said. Based on their findings, they conclude, reporting standards for AI privacy audits need to change.\u00a0<\/p>\n It gets worse, too: Patients in the dataset are generally easy to identify and, unsurprisingly, those underrepresented in medical AI training data are even easier to finger than those whose data doesn\u2019t stand out.\u00a0<\/p>\n Underrepresented groups can include those in a number of sensitive categories: Race, insurance status, sex, the protocol used to conduct medical imaging, and certain disease statuses can all function as outliers that make it easier to identify individuals.\u00a0<\/p>\n \u201cGenerally speaking, privacy risks from MIAs become more severe as a model\u2019s training cohort becomes more specific,\u201d Technical University of Munich AI in Healthcare and Medicine chair and paper lead author Moritz Knolle told The Register in an email conversation. \u201cYou could imagine \u2026 scenarios where membership in a training dataset reveals that someone has a dormant genetic condition such as Huntington’s disease, depression, or attended a specific, specialised treatment clinic.\u201d\u00a0<\/p>\n In other words, exposing healthcare AI training data could be used to identify those with sensitive health conditions, spill secrets they may not want public, or otherwise fuel discrimination.\u00a0<\/p>\n To make things even worse again, the larger the dataset, the easier it is to expose records, and \u201cthe magnitude of this change in patient-level risk was previously unknown\u201d in larger models.\u00a0<\/p>\n The privacy devil in the data details<\/p>\n This is bad and all, but it\u2019s not necessarily the end of the world, as performing an MIA attack on a medical AI model supposes the attacker already has a few things at their disposal, namely at least some medical data belonging to the people they want to identify.\u00a0<\/p>\n \u201cTo conduct a MIA an attacker needs access to a target data point,\u201d Knolle confirmed to us while also noting that their paper revealed access to a full patient data point isn\u2019t needed, in contrast to what was previously believed. \u201cIn our paper we show that an attacker with partial access can still successafully conduct MIAs.\u201d\u00a0<\/p>\n The MIA attack itself, as detailed in the paper, relies on medical AIs being more certain of their predictions if the input data is already part of their training set. A potential attacker, then, simply peppers an AI model with obtained patient data, checks the confidence level, and surmises that said patient is part of the training data.\u00a0<\/p>\n \u201cAn attacker conducting a MIA does not need to know who the data belongs to that they are trying to conduct the MIA with,\u201d Knolle explained. \u201cIn fact, all the dataset we use in our study were anonymized.\u201d<\/p>\n Anonymized in the datasets, but not the target data, that is. As explained in the paper their MIA attacks were largely error-free at the individual patient level, meaning confidence levels are an accurate way to figure out if a particular patient’s data is part of a training set.\u00a0<\/p>\n \u201cThe attacker would simply need access to someone\u2019s blood test results, or part of these results\u201d in order to infer inclusion, Knolle said.\u00a0<\/p>\n Of course, they have to get that data first, but given how<\/a> frequently<\/a> healthcare data is exposed<\/a> in breaches<\/a>, it\u2019s not exactly hard to imagine a bad actor getting ahold of something they can use.\u00a0\u00a0<\/p>\n \u201cGiven that medical data is not always securely stored it is not unthinkable that an attacker could get access, for example, by gaining unauthorized access to the database of your general practitioner after they performed a routine blood test,\u201d Knolle said.\u00a0<\/p>\n How to protect patient data?<\/p>\n Asked what he hopes this research accomplishes, Knolle told us he just wants the medical world to understand that AI training data needs to be better secured.\u00a0<\/p>\n \u201cI hope that the medical AI community will start to take privacy risks seriously and that risk mitigation techniques are used in situations where they are necessary,\u201d Knolle said.\u00a0<\/p>\n