ABSTRACT
OBJECTIVE To clarify efficacy, effectiveness, and harm of available vaccines for COVID-19, using measures in evidence-based medicine (EBM) that, in addition to relative risk reduction, consider absolute risk reduction and variations in baseline risks.
DESIGN Systematic review of studies that have considered impacts of vaccines in relation to baseline risks. Calculation of risk reduction and harms from published data in two random controlled trials and one population-based implementation study. Analysis of risk reductions in geographical areas with varying baseline risks. Comparison of results concerning COVID-19 vaccine and selected prior vaccines.
SETTING Random controlled trials of Pfizer and Moderna vaccines in multiple countries; population-based study using Pfizer vaccine in Israel. Counties with varying baseline risks in the United States; states with varying baseline risks in India.
PARTICIPANTS 43,448 and 30,420 subjects in the random controlled trials; 1,198,236 subjects in the population-based study.
INTERVENTIONS Multi-site random controlled trials of vaccine efficacy; population-based administration of vaccine with determination of effectiveness by comparison of vaccinated and unvaccinated subjects.
MAIN OUTCOME MEASURES Relative risk reduction (RRR), absolute risk reduction (ARR), number needed to be vaccinated to prevent one symptomatic infection (NNV), absolute risk of the intervention (ARI), and number needed to harm (NNH).
RESULTS A systematic review of literature in medicine and public health showed very few reports regarding ARR, NNV, ARI, and NNH; use of these indicators to compare benefits versus harms; or analysis of these EBM indicators in the context of varying baseline risks. From data in the two random controlled trials and one population-based study, calculated ARR was approximately 1 percent (as compared to RRR of 50 to 95 percent), and NNV was in the range of 100 to 500. In comparisons of ARR and NNV versus ARI and NNH, benefits and harms were not markedly different. From a sensitivity analysis of ARR and NNV in population groups with varying baseline risks, the effectiveness of vaccines as measured by ARR and NNV was substantially higher in regions with high as compared to low baseline risks. The ARR for COVID-19 vaccines was somewhat smaller and the NNV somewhat larger than achieved by some vaccines to prevent influenza and smallpox.
CONCLUSION The efficacy and effectiveness of major COVID-19 vaccines, as measured by RRR, are impressive. As measured by ARR and NNV, which take into account variation in baseline risks, the effectiveness of the vaccines is substantially higher in areas with higher baseline risks. This finding can contribute to educational efforts, informed consent procedures, and policy making about priorities for vaccine distribution, especially under conditions of access barriers linked to poverty and inequality.
WHAT IS ALREADY KNOWN ON THIS TOPIC Major COVID-19 vaccines so far have shown impressive efficacy in random controlled trials and effectiveness in population-based studies. To determine efficacy and effectiveness, these studies have used relative risk reduction (RRR), which shows the difference in event rate between those receiving and not receiving a vaccine. Reports of efficacy and effectiveness have not yet clarified other key indicators from evidence-based medicine (EBM) that consider variations baseline risks. Such indicators include measures of benefits such as absolute risk reduction (ARR) and number needed to be vaccinated (NNV), as well as measures of harm such as absolute risk of the intervention (ARI) and number needed to harm (NNH).
WHAT THIS STUDY ADDS For COVID-19 vaccines, calculated ARR is somewhat lower and NNV somewhat higher than for certain prior vaccines such as those for influenza and smallpox. Indicators of harm for COVID-19 vaccines, as measured by ARI and NNH, appear to be in the same order of magnitude as indicators of benefit such as ARR and NNV. The effectiveness of COVID-19 vaccines, as measured by ARR and NNV, is substantially higher in geographical areas with high baseline risk, compared to areas with low baseline risk.
These findings can assist in informed consent procedures, educational efforts, and priority setting in policies about distribution of vaccines, especially in the context of access barriers related to poverty and inequality.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
No specific funding was received for this study. AL and HW are semi-retired from clinical practice in internal medicine. HW receives access to library resources and personnel, internet service, and uses of other facilities as a distinguished professor emeritus at the University of New Mexico. HW obtains no ongoing salary support but does receive retirement pensions as an emeritus professor from the University of New Mexico and the University of California. AL receives a retirement pension from a private fund generated through his medical practice. Employers/sponsors had no role in the design, analysis, or dissemination of the study. The views expressed in this article are those of the authors and not necessarily those of the entities the authors are affiliated with and/or supported by.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Not applicable.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Co-author: Andrew Larkin, MD
Data Availability
All the data used in this study are publicly available and properly cited. More guided instruction to get access to the data for transparency and reproducibility will be provided on request.
