Physicians at the University of Florida noticed an uptick in sinus infections after hurricanes and turned to Epic Research for answers. The resulting analysis — drawing on millions of patient records from Epic’s massive Cosmos database — found little connection, instead attributing infections to the usual respiratory illness season.
The study, like most of Epic Research’s work, was published directly on the company’s website, not in a peer-reviewed journal. Epic Research, a subsidiary of the electronic health records giant, has built a reputation for producing rapid, accessible studies using data from more than 300 health systems.
Proponents say the scale of the Cosmos dataset allows for faster, broader analyses than traditional academic research. But some experts question whether speed comes at the expense of rigor and oversight. Without independent peer review, findings rely on Epic’s internal vetting process — a concern for researchers who see peer review as an essential safeguard.
Many in the academic research community remain distrustful of studies published outside the journal system. A recent survey of 294 corresponding authors across 28 countries who published studies in a peer-reviewed medical journal found that only about one third said they trust non-peer-reviewed research.
Arch Mainous III, PhD, vice chair for research in the Department of Community Health and Family Medicine at the University of Florida College of Medicine in Gainesville, Florida, said the peer-review system provides guardrails to make sure studies are trustworthy.
“If we say we don’t need peer review because we really want speed, you may be getting stuff out there that could be quite harmful to people,” Mainous said.
However, numerous studies have highlighted the flaws in peer review, including that it is slow, expensive, and time-consuming, fails to catch flawed and fraudulent papers, and can contain bias by geography, race, and sex.
Despite the lack of peer review, Epic’s studies, such as one showing patients with drug overdose were not being tested for fentanyl, are frequently cited in state legislation and are leading to changes in state laws in some cases, said Caleb Cox, the lead data scientist at Epic Research.
Brian Lobo, MD, professor and associate chief medical informatics officer at the University of Florida Health, who worked with Epic on the hurricane study, said investigators benefit from the scale of the dataset. The study population included residents of affected zip codes in Florida, Georgia, and South Carolina who had a healthcare encounter during the relevant study period. Lobo said that while Epic evaluated up to more than 40 million patient healthcare encounters per hurricane, he and his colleagues at the university could evaluate only about 10,000 patient encounters by themselves.
“Where I see Epic Research, and Cosmos in general, being a huge shifter in the landscape is with meta-analyses because instead of indirectly analyzing numerous studies that may have flaws or incongruent methodologies, you can now directly analyze the primary data and make more appropriate and comprehensive solutions,” Lobo told Medscape Medical News.
Epic’s Model
The research outfit created the Cosmos dataset in collaboration with health systems that use Epic software. MetroHealth in Ohio was the first to join the collaborative in 2019. Other participants include the Duke University Health System, the Yale School of Medicine, and the Baylor College of Medicine. It now includes data from 1800 hospitals and 41,000 clinics across all 50 states, as well as Canada, Lebanon, and Saudi Arabia.
Members must use Epic software so the data come into Cosmos in the same format. Health systems also must agree to contribute all their data to the Cosmos community and not sell access to it. Epic’s first study was published in April 2020. The studies are published on its website and distributed through newsletters.
Each study is conducted by two teams of in-house researchers consisting of at least one data scientist and one clinician. Epic Research employs six full-time data scientists, eight data scientists that also work on Epic software, and 23 clinicians, including physicians, nurses, and pharmacists.
The findings are vetted by researchers and clinicians who are not involved in the study in question. Cox said one of the benefits of operating outside the peer-reviewed journal system is the ability to publish findings quickly and in a highly accessible format. He said Epic researchers have accountability because they are paid to do this work as opposed to relying on volunteers for peer review.
“The disadvantage is that we don’t have someone external who is reviewing it who should be able to vouch for the work,” Cox said. “I think in this case, the trade-off is worth it for what we’re trying to accomplish, but I can understand why, for many researchers, the traditional process is more supportive of what they’re trying to accomplish.”
In choosing research topics, Epic considers whether the question can be answered with the Cosmos dataset and if the research will lead to new knowledge and has the potential to drive change in healthcare policy or clinical practice, Cox said.
For instance, an Epic Research study from 2022 in collaboration with the University of Maryland, College Park, Maryland, found just 5% of patients visiting an emergency department for an overdose were tested for fentanyl, although the drug is the primary cause of recent overdose deaths.
In 2023, Pennsylvania and Florida enacted laws requiring acute care hospitals to include fentanyl in urine drug screening. Both legislative packages referenced the Epic study, Cox said.
Rates of toxicology testing for fentanyl increased to 14% as of the end of June 2023 and to 26% for patients aged 13-17 years by the end of 2023. Epic provides updated metrics related to previously published research studies on its Epic Research Data Tracker.
Epic Research is not the only research outfit that operates outside of the peer-review world. Many nonprofit organizations, such as KFF and the Urban Institute, publish their own research.
Epic Research has teamed up with KFF on a few studies, including a 2020 analysis that showed people of color experienced higher rates of SARS-CoV-2 infection, hospitalization, and death than White patients. In another study, the two groups looked at the role telehealth played in providing mental health care during the pandemic.
Epic also partners with government organizations. It conducted a study in 2023 on the effectiveness of the vaccine JYNNEOS against mpox disease in collaboration with the CDC.
Epic’s most referenced study sought to determine if patients taking semaglutide or liraglutide would have to stay on the drugs long-term to maintain weight loss.
More than half of patients either remained around the same weight or continued to lose additional weight after discontinuing medication, while 17.7% regained all the weight they had lost or exceeded their initial weight.
At the time of the study, there wasn’t much information about weight changes a year after stopping the newer weight-loss medications, and “we weren’t sure what the real-world data would show us, but we knew it was something we had access to, would be timely, and relevant to a lot of the population,” said Kersten Bartelt, RN, researcher at the company.
Advantages and Limitations
Conducting studies from a large dataset such as Cosmos may help avoid many biases — regional, sex-based, racial — because analyzing a massive population reduces the chance that any one group will be left out, Lobo said. Analysis of subgroups could also be more reliable because the number of subjects required to measure significant outcomes in a population is often higher than most studies.
Researchers can also collaborate with Epic on studies.
“The one thing that’s exciting about Epic Research is that I think it’s opening up the ability to do big-scale data analyses to people who may not necessarily have the skills to do so by allowing partnerships and collaborations between those with data analysis skills and those who have questions or ideas,” Lobo said.
Cox said the average time for a study to be conducted and published at Epic Research is about 1 month. And the research is not behind a paywall.
“It is also in language that someone can pick up and read and understand, whether that’s a busy primary care doc who doesn’t have time to sit down and spend 15 minutes reading a journal article or a patient who is just not going to have either access to those articles that are behind a paywall or the expertise to be able to read and understand them,” Cox said.
The journal process is notoriously slow, in part because the pool of qualified reviewers is limited. Mainous said his study comparing BMI to body fat percentage took a full year to publish after submission.
Some structural limitations exist based on the de-identified nature of the Cosmos dataset. For example, in the hurricane study, researchers aggregated affected zip codes to avoid the potential of including a sample size of less than 10 people.
“I think that what we’re able to do is a great complement to what exists in peer-reviewed research and other clinical trial research going on,” Bartelt said.
Lobo, who reviews manuscripts for several journals, said although the review process reduces the chance of publishing systematically poor research or nepotism, that risk still exists.
“Moreover, with pay-for-play journals playing a larger role in scientific research, it’s becoming more challenging to differentiate between true ‘peer-reviewed’ and true ‘non-peer-reviewed’ research,” Lobo said. Under this model, authors pay an article processing charge to have their articles published.
“I think what really separates Epic Research from the rest of the non-peer-reviewed world is its commitment to methodology and its commitment to demonstrate what the data is showing, not necessarily the narrative that a particular author is trying to demonstrate. This really elevates the quality of research and insights provided by their big data analyses,” Lobo said.
Brenda Sandburg is a freelance journalist.