Scientific studies into conditions from cancer to HIV are being plagued by imposter participants, undermining the reliability of health research and potentially harming patients, Oxford academics have warned.
Researchers who use online systems to recruit participants for surveys and trials are finding that up to 94 per cent of applicants have to be disqualified as “fraudulent or suspicious” before their studies can go ahead.
This can be because they are people who are lying about having the disease being investigated, or because they are suspected to be automated bots that flood trials with fake applications.
But some of these imposters are likely to be slipping through the cracks and taking part in the studies. This would skew the results of health research, leading health policymakers to make incorrect decisions and ultimately causing harm to patients, according to a warning published in the BMJ.
“The undetected presence of imposter participants in quantitative datasets threatens the integrity of health research and, by extension, the policies and clinical decisions built on it,” wrote the academics, led by Eileen Morrow, a doctoral research fellow at the Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences at the University of Oxford. “Results clouded by imposter participants may affect clinical treatment.”
Academics remain mystified as to the motivation behind the imposters. The most likely driver for many is financial, when there is a study that pays participants for their time.
But analysis suggests that the same problem is seen with studies that offer no cash incentives, with academics suggesting: “Proposed alternatives include boredom, curiosity or even an ideological intent to disrupt research.”
A review published in the journal BMJ Evidence-Based Medicine looked at 23 studies and found that 18 had reported “encountering fraudulent responses”.
Most were studies that investigated specific behaviours or groups of people, including users of alcohol, tobacco, opioids and other drugs, cancer survivors, Covid-19 patients and people living with HIV, the review found.
An online survey investigating how ovarian cancer patients rated the communication from their doctors had 576 applications to take part. Most came in between midnight and 4am.
“The authors judged 94 per cent of responses to be fraudulent and the remaining 6 per cent suspicious, with no participant deemed unquestionably legitimate,” the academics said. “As a result, they closed and relaunched the survey with stricter protocols.”
Most drug trials are likely to be more secure as patients are recruited via clinics and verified by doctors
SIMPLEIMAGES/GETTY IMAGES
The problem was not unique to surveys. A randomised controlled trial, the gold standard for making objective assessments in science, into the effectiveness of an app to help people reduce their alcohol intake found that 76 per cent of online applicants were bots. A further 4 per cent were “deceptive human respondents”.
Morrow said that she had once interviewed a person for a study who had claimed to be a UK healthcare professional. When asked to prove their membership of a professional body they “abruptly ended the call”. They then enrolled again under the same name and same email address, “this time claiming to be a child who had undergone surgery”.
Eileen Morrow said that imposter participants were putting the research behind clinical decisions at risk
GETTY IMAGES
She said that most drug trials were “relatively secure” as participants tended to be recruited via clinics and had their diagnoses verified by doctors, but survey-style studies that collect data about patients’ experiences or test health apps were more at risk.
Morrow said she was not aware of study papers having to be retracted after publication due to imposter participants but warned: “Unless researchers notice anomalies in their datasets they may never think to look for imposters, and most detection methods are unverified.”
• NHS app will recruit millions for clinical trials to fight cancer
The academics wrote: “Health research now faces the same risks of fraud that have plagued non-health spaces such as market research. However, the stakes are higher; health research informs clinical decisions, service design, and resource allocation.”
They said researchers should ensure that detecting imposters was a key part of designing studies and that all research should make clear what safeguards are used.
Some researchers use tools to detect whether an online survey was filled out suspiciously quickly, suggesting a bot may have been involved. Others identify “implausible text patterns”, ask applicants to submit videos, or use automated “Turing tests” that can differentiate between human and computerised respondents.
The academics wrote of imposter participants: “Their effect is demonstrable and their detection inconsistent. In an age where online recruitment underpins everything from randomised controlled trials to surveys, they risk undermining the integrity of health research and the decisions built on it.”