While this may seem like an esoteric topic, it sheds light on the ongoing disruption within the agency over vaccine policy. CDC leadership has already blocked publication of its own staff\u2019s study on Covid vaccine effectiveness, citing methodological concerns. And just this week, Health and Human Services posted a new charter for the Advisory Committee on Immunization Practices<\/a> \u2014 the panel that advises CDC on vaccine recommendations \u2014 that diminishes its role in evaluating vaccines and instead directs it to assess \u201calternatives\u201d to vaccination. The methods used to measure whether vaccines work, and the way they were discussed during the recent seminar, are not a side issue. They are squarely in the crosshairs.<\/p>\n In his opening remarks, Jay Bhattacharya (who is leading the agency though not officially acting director), framed the series as a reflection of CDC\u2019s renewed commitment to the principles driving scientific progress: excellence, collaboration, transparency, intellectual honesty, and willingness to engage in thoughtful debate on complex issues. Great!\u00a0<\/p>\n The specific topic was how to measure the degree to which vaccines protect people in the real world \u2014 so-called vaccine effectiveness studies.<\/p>\n \t\t\t \t\t\t\t\t\tNew ACIP charter broadens criteria for members, calls for review of alternatives to vaccines<\/a><\/p>\n In fact, the discussion went deeper than that, hinging on a debate about a specific study design. The panel included three scientists with deep expertise in vaccine effectiveness methodology: Carrie Reed, chief of CDC\u2019s Influenza Epidemiology and Prevention Branch; Emily Martin, professor at the University of Michigan; and Natalie Dean, associate professor at Emory University\u2019s Rollins School of Public Health; as well as a representative from HHS leadership, Martin Kulldorff, chief science officer for the Office of the Assistant Secretary for Planning and Evaluation.\u00a0<\/p>\n Reed and Martin presented, in depth, the principles of real-world vaccine studies and the methods underlying them. They laid out the fundamentals of the study designs, the reasons we use them, the real-world questions they have answered, the policy decisions they have informed, and, critically, their limitations, as well as the decades of work that have gone into examining how significant those limitations are and what can be done to address them.<\/p>\n While there are many approaches to estimating vaccine effectiveness, the test-negative design, or TND, is the mainstay. It\u2019s a form of a case-control study, where you compare people who had a disease (cases) to those who didn\u2019t (controls). The innovation of the TND is that cases and controls are identified based on whether they test positive or negative for a particular vaccine-preventable pathogen.<\/p>\n For example, if you\u2019re hospitalized with a severe respiratory infection and test positive for flu, you\u2019re a case. If you test negative, you\u2019re a control. Then you look at whether those people were vaccinated. If the vaccine is working, cases will be less likely to have been vaccinated than controls.\u00a0<\/p>\n It\u2019s an elegant solution to a real-world problem. It\u2019s fast, inexpensive, and widely used. Of course, it\u2019s not perfect, which is precisely why the scientific community has spent years working to validate and improve it. That\u2019s how applied science works.<\/p>\n Kulldorff claimed on the Grand Rounds to have identified \u201cthe fatal flaw in test-negative designs.\u201d One of his stated fatal flaws was that you can\u2019t use this approach when cases and controls have different diseases \u2014 you shouldn\u2019t use it to compare risk factors for pancreatic versus lung cancer, for example. To which the response of the other panelists \u2014 and, I\u2019m imagining, my entire field watching the webcast \u2014 was essentially:\u00a0Of course, and that\u2019s why we don\u2019t.\u00a0<\/p>\n The TND relies on people seeking care for the same illness, like symptoms of a respiratory infection. They don\u2019t seek care for a specific pathogen. But as Dean pointed out, vaccines prevent specific pathogens: Covid vaccines prevent SARS-CoV-2, but not RSV or flu \u2014 and that\u2019s precisely why the design works.<\/p>\n So why does any of this matter? It matters on two levels. First, practically: If the TND is discredited or sidelined, we lose one of our primary tools for monitoring vaccine performance in real time. This is the kind of ongoing surveillance analysis that tells us whether this season\u2019s flu shot is working or if a new vaccine, like for RSV, is performing as expected.\u00a0<\/p>\n Second, politically:\u00a0Bhattacharya has already blocked publication of a CDC study on Covid vaccines<\/a>, citing \u201cconcerns regarding the methodological approach to estimating vaccine effectiveness.\u201d (Notably, the paper was ultimately published in a peer-reviewed journal<\/a>.)\u00a0 Others have written extensively<\/a> about the inconsistency and lack of logic in that decision, so I won\u2019t here.<\/p>\n More broadly, Kulldorff, Bhattacharya, and other appointees in Robert F. Kennedy Jr.\u2019s HHS have raised persistent doubts about Covid vaccines and about vaccine science more generally. What happened at the Grand Rounds appeared to be an attempt to manufacture a scientific debate where, in reality, there isn\u2019t one.<\/p>\n The TND works quite well. Some of the world\u2019s foremost experts in TND and vaccine evaluation are at the CDC. They have innovated impressively with the constraints and resources and data available to them, and they constantly work to validate and improve their methods. Any open-minded observer would have come away from the discussion reassured that these scientists and the broader network of investigators who seek to answer questions about vaccine performance in the real world are rigorous, creative, and methodical.<\/p>\n Bhattacharya\u2019s closing remarks included the platitudes: \u201cWe owe each other this kind of discussion\u201d and \u201cwe owe the public the very best regarding the truth.\u201d<\/p>\n However, a more pointed truth emerged from the recent discussion: We have robust tools for evaluating vaccines in the real world and a community of committed scientists who dispassionately do the work. They don\u2019t claim their methods are perfect; they\u2019ve spent careers proving they aren\u2019t, and improving them anyway, and they will continue to do so.<\/p>\n The question now is whether the institutions that depend on that work will trust it.<\/p>\n Ben Lopman is professor of epidemiology and global health at Emory University\u2019s Rollins School of Public Health.<\/p>\n","protected":false},"excerpt":{"rendered":"What happens if public health agencies stopped trusting their own tools, scientists, and surveillance networks for measuring whether…\n","protected":false},"author":2,"featured_media":529236,"comment_status":"","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[10],"tags":[6904,103,61,60,2361,89,11987],"class_list":["post-529235","post","type-post","status-publish","format-standard","has-post-thumbnail","category-health","tag-cdc","tag-health","tag-ie","tag-ireland","tag-public-health","tag-research","tag-vaccines"],"_links":{"self":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/529235","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/comments?post=529235"}],"version-history":[{"count":0,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/posts\/529235\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media\/529236"}],"wp:attachment":[{"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/media?parent=529235"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/categories?post=529235"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.newsbeep.com\/ie\/wp-json\/wp\/v2\/tags?post=529235"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}![\"\"](\"https:\/\/www.newsbeep.com\/ie\/wp-content\/uploads\/2026\/07\/GettyImages-2249461848-768x432.jpg\")
\t\t<\/p>\n