Cherry Picking: The claim overlooked the fact that other pieces of data, including some included in the same CDC presentation, actually indicated that COVID-19 vaccines reduced the risk of COVID-19 hospitalization.
FULL CLAIM: “vaccinated and boosted people were MORE likely to be hospitalized with the new Omicron variant than unvaccinated people”
REVIEW
Casting doubt on the effectiveness of COVID-19 vaccines has been a recurring theme of misinformation since the vaccination rollout in December 2020. Health Feedback has covered this subject in several previous reviews, explaining why claims that vaccinated individuals are more at risk of getting infected, getting hospitalized or dying from COVID-19 were misleading or inaccurate.
Writer Alex Berenson made a similar claim in September 2023 in a post on Substack. In his post, Berenson claimed that “vaccinated and boosted people were MORE likely to be hospitalized with the new Omicron variant than unvaccinated people” according to data from the U.S. Centers for Disease Control and Prevention (CDC).
Berenson previously propagated misinformation about COVID-19 and COVID-19 vaccines in the past. Here again, his claim is inaccurate and the CDC data he used as supporting evidence actually don’t support the claim. We explain why below.
Berenson based his claim on a presentation about COVID-19 vaccine effectiveness (VE) at the 12 September 2023 meeting of the CDC’s Advisory Committee on Immunization Practices (ACIP). Vaccine effectiveness (VE) measures the ability of the vaccine to reduce a specific outcome, such as infection, hospitalization or death. A positive VE means that the vaccine effectively lowers the risk of that outcome. In theory, a negative VE would mean that the vaccine increases that risk.
The CDC presentation reported a positive VE in all cases except under a specific set of conditions (slide 25 of the presentation). The reported VE was indeed seemingly negative when all the following conditions were satisfied:
- The data came from the IVY vaccine surveillance network
- The XBB.1.5 variant was predominant in the U.S.
- People had only received the monovalent vaccine that targeted the original SARS-CoV-2, or had also received the bivalent booster targeting both the original SARS-CoV-2 and the Omicron variant three to six months before hospital admission.
The negative VE data forms the basis for Berenson’s claim. However, this interpretation of the data is erroneous because it leaves out some crucial caveats.
First, the presentation reported the average VEs together with their 95% confidence intervals (CI). For people who only received the monovalent vaccine, the average VE is -10% but its confidence interval ranges from -35% to +10%. For people who received a bivalent booster between three and six months prior hospitalization, the average VE is -7% but the confidence interval ranges from -41% to +19%.
To understand why this is important, we must first understand that the VE data were obtained from a sample of the population, not the entire vaccinated population in the U.S. Therefore, the calculated VE is in fact an estimate of the true VE that we would find if we were able to include the entire population in the analysis.
A 95% CI tells us we are 95% confident that within the confidence interval sits the true VE. More specifically, it means that if we were to repeat the same study 100 times, each time calculating a new 95% CI, the true VE would be included in those CIs 95% of the time[1].
Going back to the actual numbers from the CDC presentation, the -35% to 10% CI interval in the case of people who only received the monovalent vaccine tells us that we are 95% confident that the actual VE lies somewhere between -35% and +10%. In other words, this piece of data alone cannot allow us to establish that the true VE is negative or positive.
Second, the CDC presentation contains a warning with respect to the VE estimates for people who received the bivalent booster:
“These estimates are imprecise, which might be due to there being a relatively small number of persons in each level of vaccination or cause status. This imprecision indicates that the actual VE could be substantially different from the point estimate shown, and estimates should therefore be interpreted with caution.”
Therefore, the VE data for people who received a booster cannot be used to conclusively establish the effect of vaccination on hospitalization, contrary to Berenson’s post.
Last, it is also important to highlight that the CDC presented other data, obtained from the VISION vaccine effectiveness surveillance network, showing a positive VE in all cases, including during the XBB.1.5 period (slide 27 of the presentation).
Therefore, the CDC’s data presented during the ACIP meeting of 12 September 2023 don’t support Berenson’s claim. Statistical uncertainty limits the reliability of the results, notably in the case of the VE calculated from the IVY network.
Furthermore, data obtained from the VISION network actually contradicted the claim. Yet, Berenson only reported the data from the IVY network, and not from VISION, which is suggestive of cherry-picking data. We reached out to Berenson to ask if he was aware of these statistical limitations and additional data; we’ll update this review if new information becomes available.
Finally, researchers from the University of North Carolina also reported positive, albeit low, VE against hospitalization, even when the XBB Omicron lineage was predominant[2].
REFERENCES
- 1 – Tan & Tan (2010) The Correct Interpretation of Confidence Intervals. Proceedings of Singapore Healthcare.
- 2- Lin & Sunny (2023) Durability of Bivalent Boosters against Omicron Subvariants. The New England Journal of Medicine.
