Abstract
Background Estimating real-world vaccine effectiveness is vital to assess the impact of the vaccination programme on the pandemic and inform the ongoing policy response. However, estimating vaccine effectiveness using observational data is inherently challenging because of the non-randomised design and the potential for unmeasured confounding.
Methods We used a Regression Discontinuity Design (RDD) to estimate vaccine effectiveness against COVID-19 mortality in England, exploiting the discontinuity in vaccination rates resulting from the UK’s age-based vaccination priority groups. We used the fact that people aged 80 or over were prioritised for the vaccine roll-out in the UK to compare the risk of COVID-19 and non-COVID-19 death in people aged 75–79 and 80–84.
Findings The prioritisation of vaccination of people aged 80 or above led to a large discrepancy in vaccination rates in people 80–84 compared to those 75–79 at the beginning of the vaccination campaign. We found a corresponding difference in COVID-19 mortality, but not in non-COVID-19 mortality, suggesting that our approach appropriately addresses the issue of unmeasured confounding factors. Our results suggest that the first vaccine dose reduced the risk of COVID-19 death by 52.6% (95% Cl 26.6–84.2) in those aged 80.
Interpretations Our results support existing evidence that a first dose of a COVID-19 vaccine has a strong protective effect against COVID-19 mortality in older adults. The RDD estimate of vaccine effectiveness is comparable to previously published studies using different methods, suggesting that unmeasured confounding factors are unlikely to substantially bias these studies.
Funding Office for National Statistics.
Evidence before this study We searched PubMed for studies reporting on the ‘real-world’ effectiveness of the COVID-19 vaccination on risk of death using terms such as “COVID-19”, “vaccine effectiveness”, “mortality” and “death”. The relevant published studies on this topic report vaccine effectiveness estimates against risk of death ranging from 64.2% to 98.7%, for varying times post-vaccination. All of these are observational studies and therefore potentially subject to bias from unmeasured confounding. We found no studies that used a quasi-experimental method such as regression discontinuity design, which is not subject to bias from unmeasured confounding, to calculate the effectiveness of the COVID-19 vaccination on risk of COVID-19 death, or on other outcomes such as hospitalisation or infection.
Added value of this study The estimates of vaccine effectiveness based on observational data may be biased by unmeasured confounding. This study uses a regression discontinuity design to estimate vaccine effectiveness, exploiting the fact that the vaccination campaign in the UK was rolled out following age-based priority groups. This enables the calculation of an unbiased estimate of the effectiveness of the COVID-19 vaccine against risk of death.
The vaccine effectiveness estimate of 52.6% (95% Cl 26.6–84.2) is slightly lower but similar to previously published estimates, therefore suggesting that these estimates are not substantially affected by unmeasured confounding factors and confirming the effectiveness of the COVID-19 vaccine against risk of COVID-19 death.
Implications of all the available evidence Obtaining an unbiased estimate of COVID-19 vaccine effectiveness is of vital importance in informing policy for lifting COVID-19 related measures. The regression discontinuity design provides confidence that the existing estimates from observational studies are unlikely to be substantially biased by unmeasured confounding.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This research was funded by the Office for National Statistics.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethical approval was obtained from the National Statistician's Data Ethics Advisory Committee (NSDEC(20)12)
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
Updated the method used to estimate vaccine effectiveness based on the outputs of the regression discontinuity design
Data Availability
The ONS COVID-19 Public Health Linked Data Asset is available on the ONS Secure Research Service for Accredited researchers. Researchers can apply for accreditation through the Research Accreditation Service. The derived data used to estimate the RDD will be made publicly available.
