ABSTRACT
With data from 2 US claims databases (Optum, CVS Health) supplemented with Immunization Information System COVID-19 vaccine records, we evaluated overall and time-specific vaccine effectiveness (VE) of an initial primary series for 3 monovalent COVID-19 vaccines—BNT162b2, mRNA-1273, and JNJ-7836735—in adults (18-64 years). Vaccinated individuals were matched to unvaccinated comparators, and we estimated VE against any medically diagnosed COVID-19 and hospital/emergency department (ED)-diagnosed COVID-19. Additionally, we estimated VE by era of predominant variants, in subgroups, and compared across vaccine brands. The cohorts consisted of 341,097 (Optum) and 1,151,775 (CVS Health) matched pairs for BNT162b2; 201,604 (Optum) and 651,545 (CVS Health) for mRNA-1273; and 49,285 (Optum) and 149,813 (CVS Health) for JNJ-7836735. Summary VE estimates from meta-analysis against hospital/ED-diagnosed COVID-19 were: BNT162b2, 77% (95% CI, 76%-78%); mRNA-1273, 84% (95% CI, 83%-85%), JNJ-7836735 66% (95% CI, 63%-68%). VE estimates were higher for hospital/ED-diagnosed COVID-19 than for medically diagnosed COVID-19, and VE estimates were highest in adults receiving mRNA-1273 for both outcomes. VE was sustained for approximately 7 months for medically-diagnosed and up to 9 months for hospital/ED-diagnosed COVID-19. VE differed by brand and variant era. Additional real-world research is needed as new variants and recommendations for updated vaccines have evolved.
INTRODUCTION
Vaccines for the prevention of coronavirus disease 2019 (COVID-19) became available in the United States (US) through either emergency use authorization or licensure by the US Food and Drug Administration (FDA). This study evaluated the initial, monovalent primary series of the first 3 authorized vaccines in the US—BNT162b2 (Pfizer-BioNTech’s messenger ribonucleic acid [mRNA] COVID-19 vaccine, Comirnaty®), mRNA-1273 (Moderna’s mRNA COVID-19 vaccine, Spikevax®), and JNJ-7836735 (Janssen Pharmaceutical Company’s adenovirus COVID-19 vaccine). These vaccines demonstrated efficacy in preventing various COVID-19 outcomes, including symptomatic infection,1–3 severe COVID-19,1–4 and death due to COVID-194 in clinical trials before authorization.
Many early real-world US studies affirmed the real-world effectiveness of COVID-19 vaccines in preventing severe disease,5–12 though some suggested that vaccine effectiveness (VE) waned over time in fully vaccinated individuals.3,7,13–15 However, some studies demonstrated only minimal or no meaningful decreases in VE estimates over time against more severe COVID-19 or COVID-19-related hospitalization.5–7,9,16 In the context of rapidly evolving pandemic conditions, real-world studies on the duration of vaccines’ effectiveness may be challenging. Approaches that compare individuals with different lengths of follow-up may be subject to selection bias and time-related biases (e.g., immortal person-time bias).17 Changes in the predominant circulating viral variants further complicate the question of waning effectiveness over time. Specifically, the emergence of Delta and Omicron variants, which emerged several months after vaccination rollout, resulted in large increases in COVID-19 cases among unvaccinated and vaccinated individuals in the US.18
FDA continues to monitor the safety and effectiveness of COVID-19 vaccines using real-world evidence through the Biologics Effectiveness and Safety (BEST) Initiative. This study, which is part of the BEST Initiative, evaluated the potential for changing effectiveness of the original complete COVID-19 primary series vaccine over time using rich real-world data and robust methodology.
MATERIALS AND METHODS
Data Source and Population
This cohort study of adults aged 18-64 years was conducted using 2 US health insurance claims data sources (Optum pre-adjudicated commercial claims; CVS Health adjudicated commercial claims) supplemented with vaccination records from US Immunization Information Systems (IISs) (sTable 1).19 The data sources contain enrollment information, dispensed prescription drug claims, and hospital, physician, and healthcare professional health insurance claims for individuals with commercial insurance from throughout the US. The study period began on 11 December 2020 (the date COVID-19 vaccines were first available in the US) and ended on 15 January 2022 in Optum and 31 March 2022 in CVS Health (end of complete data at the time of analysis). Vaccinated individuals were identified at the first record of a COVID-19 vaccination during the study period (Time 0), and unvaccinated individuals were matched by calendar date, US county of residence, age (5-year increments), sex, immunocompromised status, pregnancy status, history of COVID-19 diagnosis, influenza vaccination, and presence of at least 1 of the conditions identified by CDC as increasing individuals’ risk of severe COVID-19.20 The calendar date of the vaccinated individual were set at Time 0 for the matched unvaccinated comparator.
Vaccinated and matched unvaccinated individuals were required to meet the following eligibility criteria on or before Time 0: have continuous medical and pharmacy coverage in the database before Time 0 for at least 365 days prior to Time 0 or back until 11 December 2020 (the day COVID-19 vaccines were initially authorized in the US); be within the indicated/authorized age range of the vaccine; and reside within the catchment area of the combined claims and IIS data. To align vaccinated and unvaccinated individuals, all eligible individuals were required to be free of any of the following conditions in the specified time periods before Time 0, given that these could reduce the probability of vaccination: monoclonal antibody or convalescent plasma treatment (90 days); COVID-19 diagnosis (30 days); fever, nausea/vomiting, rash diagnosis (3 days); hospitalization or emergency department (ED) visit (3 days); or be hospitalized or a long-term care resident (on Time 0).21
Exposure Assessment
Vaccine doses were identified in insurance claims and IIS databases using procedure billing codes for vaccine administration, National Drug Codes for vaccine products, or IIS administration records.22 The dose number was inferred from the order of observed doses within an individual’s record—thus continuous enrollment for the entire period since the introduction of vaccines was required. If a vaccine dose of the same brand or a dose record which did not specify the vaccine brand occurred within 3 days following another dose, the second record was considered a duplicate and was excluded; if a dose for a different brand was received within 3 days, the brand of the first dose was considered unknown and both dose records were excluded.
All subsequent COVID-19 vaccines received by an individual after Time 0 were used to evaluate continuing adherence to the primary series (Figure 1). For vaccines with 2-dose primary series, Dose 2 needed to be received no more than 4 days before the recommended interval and no more than 42 days after Dose 1 (sTable 2). A secondary analysis evaluated receipt of only a single dose of the 2-dose mRNA vaccine primary series (Supplemental Methods).
Outcome Assessment
Individuals were followed from Time 0 until the occurrence of an outcome of interest, or were censored at the first of the following: last day of the study period; health plan disenrollment; relocation outside of the claims and IIS data catchment area; death; deviation from receiving the primary series (i.e., receipt of Dose 2 too early, failure to receive Dose 2 on time, receiving a booster/additional dose) for the vaccinated, or receipt of any COVID-19 vaccine for the unvaccinated.
Two nested COVID-19 outcomes were evaluated separately: (1) medically diagnosed COVID-19, identified as a recorded COVID-19 diagnosis from hospital, ED, outpatient, or physician encounters; and (2) hospital/ED-diagnosed COVID-19. Recorded COVID-19 diagnosis codes (ICD-10-CM U07.1) were identified in claims in any diagnosis field coding position. For each outcome, the date of the first observed qualifying diagnosis was assigned as the outcome date.
Covariates
Descriptive characteristics were measured on or before Time 0 (Figure 2) using enrollment, diagnosis, procedure, and pharmacy data.23 Covariates included demographics, comorbidities, frailty indicators, healthcare utilization, and conditions potentially increasing risk of severe COVID-19.20
Statistical Analysis
Analyses were performed separately by data source and vaccine brand. The distribution of characteristics by vaccination group were described with means, standard deviations (SDs), medians, and first and third quartiles (Q1, Q3) for continuous variables, and counts and proportions for categorical variables. Covariate balance between vaccination groups was evaluated with absolute standardized differences.24
In addition to individual matching, propensity score weighting was used to address confounding. Propensity scores were estimated with multivariable logistic regression models including all prespecified covariates and matching factors (sFigure 1). Stabilized inverse probability of treatment (sIPT) weights were estimated from propensity scores with truncation below the 1st percentile and above the 99th percentile of the propensity score distribution.
The cumulative incidence of each COVID-19 outcome was estimated in the sIPT-weighted vaccine exposure groups as 1 minus the Kaplan-Meier estimator.25 Cumulative incidence of COVID-19 outcomes in the first 14 days of follow-up was evaluated as a negative control outcome because vaccines are not expected to produce an immune response until 10-14 days after vaccination.26,27 Hazard ratios (HRs) for the association of vaccination status with COVID-19 outcomes were estimated using sIPT-weighted Cox proportional hazards models; 95% confidence intervals (CIs) were estimated with robust sandwich variance estimators.28 Overall VE was estimated as 1 minus the HR. Time-specific VEs were estimated using daily cumulative incidence estimates in both treatment groups to estimate daily risk ratios (RRs) from which VEs were estimated as 1 minus the RR; time-specific risk differences (RDs) were estimated by subtracting daily cumulative incidence estimates. Nonparametric bootstrapping was used to estimate 95% CIs for time-specific VEs and RDs.
Meta-analyses across data sources were performed with fixed-effects meta-analysis methods29 for the overall, subgroup, and variant era–specific VE. Statistical evidence of heterogeneity between data source estimates was evaluated by obtaining log-transformed HR estimates and their standard errors, with p values less than 0.05 indicating evidence of statistical heterogeneity.
Subgroup analyses for the overall VE estimation were performed by age, immunocompromised status, and history of COVID-19 diagnosis. Analyses by variant era (pre-Delta era, 11 December 2020 through 31 May 2021; Delta era, 1 June 2021 through 24 December 2021; Omicron era, 25 December 2021 through the end of data availability30) were restricted to individuals with Time 0 within the era, and follow-up was censored on the last day of the era. Two sensitivity analyses evaluated the potential selection bias introduced by censoring for receipt of a censoring dose (e.g., individuals in the unvaccinated group receiving any vaccine, or individuals in the vaccinated group receiving Dose 2 too early, Dose 2 of a different brand, or a third dose): (1) delay censoring for 7 days after receipt of a censoring vaccine dose, as there would not be an expected effect during this time; and (2) apply inverse probability of censoring weights.
Quantitative bias analyses (QBA)31,32 estimated the impact of potential exposure misclassification because of missing vaccine records by estimating corrected HRs accounting for a range of vaccine exposure sensitivities (Supplemental Methods).
A secondary analysis evaluated the relative VE (RVE) of receiving a complete primary series of different COVID-19 vaccine brands. For each comparison, the study period was restricted to calendar time periods when both vaccine brands in that comparison were authorized. To account for variation in local COVID-19 burden between the 2 exposure groups, vaccinated individuals in both exposure groups were exact-matched 1:1 without replacement on calendar week, age, sex, and US county of residence. RVE was estimated as 1 minus the HR for each pairwise comparison.
Statistical analyses were performed with SAS Version 9.4 (SAS Institute Inc., Cary, NC) and R 4.1.2 (R Foundation for Statistical Computing, Vienna, Austria). This activity was conducted as part of the FDA public health surveillance mandate; FDA did not require Institutional Review Board review. Individual-level consent was not required for this analysis of secondary healthcare data. The study protocol was publicly posted on the BEST Initiative webpage.23
RESULTS
Descriptive Analyses
In Optum, we identified 612,125 individuals who received an eligible first COVID-19 vaccine dose during the study period. After matching, the analytic cohorts consisted of 341,097 in each group for BNT162b2, 201,604 in each group for mRNA-1273, and 49,285 in each group for JNJ-7836735. In CVS Health, we identified 1,979,109 eligible individuals receiving a first COVID-19 vaccine dose during the study period. After matching, the analytic cohorts consisted of 1,151,775 in each group for BNT162b2, 651,545 in each group for mRNA-1273, and 149,813 in each group for JNJ-7836735. The length of follow-up varied across vaccine brands and outcome analyses, but in Optum, median follow-up ranged from 232 to 248 days for the vaccinated groups, and 88 to 114 days for the unvaccinated; in CVS Health, median follow-up ranged from 244 to 258 days in the vaccinated, and from 102 to 129 days in the unvaccinated.
Selected characteristics of individuals who received a COVID-19 vaccine dose and matched unvaccinated comparators are shown in Table 1 (complete details in Supplemental Information sTables 3-5). Across both data sources, the mean age was approximately 42 to 43 years for each vaccine-specific cohort. The BNT162b2 and mRNA-1273 matched cohorts were slightly more female (51%-52%), but the JNJ-7836735 matched cohort was slightly more male (56%). A very small proportion of the cohorts were pregnant at the time of vaccination (approximately ≤ 1%), and approximately 4% of each cohort were immunocompromised. Vaccine and comparator groups were matched exactly on age, sex, geography, pregnancy, COVID-19 history, and immunocompromised status, thus these characteristics were all perfectly balanced in the matched groups. Additionally, all other measured characteristics (i.e., healthcare utilization, comorbidities, frailty markers) were all well balanced between treatment groups as indicated by absolute standardized differences close to 0 (sTable 3 through sTable 5). The substantial overlap of propensity score distributions for all cohorts indicated a high degree of exchangeability between vaccinated and unvaccinated groups for all measured covariates (sFigure 1).
Outcome Analyses
In each cohort, incidence of medically diagnosed COVID-19 was much higher than the incidence of hospital/ED-diagnosed COVID-19. For instance, in Optum, the BNT162b2 vaccinated group had 11,399 medically diagnosed cases and 1,066 hospital/ED-diagnosed cases in nearly 70 million person-days of follow-up; in CVS Health, the BNT162b2 vaccinated group had 40,116 medically diagnosed cases and 4,496 hospital/ED-diagnosed cases in nearly 270 million person-days of follow-up (sTable 6).
Overall Vaccine Effectiveness
Overall VE estimates are summarized in Figure 3 (complete details in sTable 6). Summary VE estimates from meta-analysis against medically diagnosed COVID-19 for receiving the complete primary series compared with being unvaccinated ranged from 38% to 59% (BNT162b2, 50% [95% CI, 50%-51%]; mRNA-1273, 59% [95% CI, 58%-60%]; JNJ-7836735, 38% [95% CI, 36%-40%]). Meta-analysis VE estimates against hospital/ED-diagnosed COVID-19 were higher, ranging from 66% to 84% (BNT162b2, 77% [95% CI, 76%-78%]; mRNA-1273, 84% [95% CI, 83%-85%]; JNJ-7836735, 66% [95% CI, 63%-68%]). For all outcomes and vaccines, Optum VE estimates were slightly higher than those in CVS Health. Because of the precision of the study estimates, most estimates of overall VE demonstrated evidence of statistical heterogeneity, but estimated magnitudes were similar clinically; therefore, summary VE estimates were reported for all estimates as summaries across both data sources, despite evidence of statistical heterogeneity.
In subgroup analyses, VE estimates for those with a previous COVID-19 diagnosis were generally lower than in those without a previous COVID-19 diagnosis for all brands and both outcomes in both data sources (sFigure 2); however, in the unvaccinated groups, COVID-19 incidence was generally lower among those with a previous COVID-19 diagnosis than among those without a previous diagnosis (sTable 7). Sensitivity analyses were mostly consistent with the primary analyses, but the IPC-weighted analyses resulted in slightly higher VE estimates for the medically diagnosed COVID-19 outcome (sFigure 3). QBA for potentially missing vaccine records suggest the observed overall VE estimates may underestimate the true VE by 2% to 14%, depending on the outcome, analysis, and data source (sTable 8).
Vaccine Effectiveness Over Time
To evaluate potential changing VE over time for the primary vaccinated versus unvaccinated comparison, the cumulative incidence of COVID-19 outcomes over time by vaccination status and vaccine brand was plotted (sFigure 4). Because of its introduction later in the study period, there was less available follow-up time for the JNJ-7836735 vaccine than the other available vaccines. For each COVID-19 vaccine comparison and outcome, rates were higher in the unvaccinated group throughout follow-up. Generally, differences in outcome rates between the 2 groups were initially small but then widened over time.
Daily VE estimates throughout follow-up were estimated, and the VE estimates over time were plotted (Figure 4; VE and RD estimates and 95% CIs at specific timepoints are reported in sTable 9). Estimates fluctuated widely during the first week after Time 0 (as the early time periods have the smallest number of events, and estimates were highly subject to change), and negative control outcome analyses of the first 14 days revealed some small differences between vaccinated and unvaccinated groups, mainly in the first few days immediately after vaccination (sFigure 5). Following the initial few days after vaccination, all vaccines followed similar patterns: for medically diagnosed COVID-19, VE estimates increased over time until approximately 90 days and were sustained through approximately day 183, with evidence of waning afterward by day 270; for hospital/ED-diagnosed COVID-19, the pattern was very similar, but the VE estimates were higher throughout, and a smaller degree of waning was observed by day 270 (Figure 4, sTable 9). Despite evidence of waning of the VE estimate, absolute RD estimates showed increasing reductions in numbers of cases through at least day 270 after vaccination (sTable 9).
Vaccine Effectiveness by Variant Era
VE against both COVID-19 outcomes generally declined between the pre-Delta and Omicron eras (sFigure 6), although there was less available follow-up time during the Omicron era (e.g., only 20 days of follow-up in Optum occurred during the Omicron era), resulting in imprecise estimates. The change in VE between the pre-Delta and Delta eras was greatest for medically diagnosed COVID-19; the meta-analyzed VE of JNJ-7836735 declined by 12 percentage points, followed by BNT162b2 with a 3–percentage point decline. In contrast, the VE of mRNA-1273 increased by 4 percentage points between the pre-Delta and Delta period. Much larger decreases in VE were observed between the Delta and Omicron eras (sFigure 6). For hospital/ED-diagnosed COVID-19, a similar pattern was observed, with only modest changes between the pre-Delta and Delta eras, and more substantial decreases with the Omicron era.
Relative Vaccine Effectiveness
When estimating the overall RVE for each vaccine brand during time periods when both brands were available, the summary VE estimates for mRNA-1273 demonstrated 16% (95% CI: 15%, 18%) to 19% (95% CI: 14%, 24%) higher effectiveness than BNT162b2 for medically diagnosed and hospital/ED-diagnosed COVID-19, respectively. The relative effectiveness of the 1-dose JNJ-7836735 series was lower than both 2-dose mRNA vaccine brands for both outcomes (sFigure 7).
Receiving Only One Dose
For the analysis of receiving only a single dose of a 2-dose primary series compared with being unvaccinated, meta-analyzed VE estimates for hospital/ED-diagnosed COVID-19 were 57% (95% CI: 54%, 59%) and 69% (95% CI: 67%, 72%) for single doses of BNT162b2 and mRNA-1273, respectively (sFigure 8). However, these VE estimates for both outcomes were more modest than those seen for the analyses of the complete 2-dose primary series.
DISCUSSION
In this large, real-world evaluation, receiving a complete primary series of COVID-19 vaccination was effective at preventing medically diagnosed COVID-19 and hospital/ED-diagnosed COVID-19 in adults aged 18 to 64 years, and that effectiveness was sustained for at least 7 months. The observed VE estimates were generally higher for hospital/ED-diagnosed COVID-19 than for any medically diagnosed COVID-19 and were highest in adults receiving the mRNA-1273 vaccine across all analyses. Results from both data sources suggested lower VE in adults receiving the JNJ-7836735 vaccine.
VE was also reduced during the Omicron era across all brands, although sample sizes were too small for the hospital/ED-diagnosed COVID-19 results to be meaningful. When evaluating VE by variant era, VE estimates were lower in the Omicron era than in the Delta or pre-Delta eras for all vaccines and both outcomes. Although waning of VE for the primary series and lower VE in later variant eras was observed, booster doses were recommended and have also been demonstrated to be effective.33
Our study utilized 2 US data sources, representing commercially insured adults from jurisdictions across the US with participating IISs who received COVID-19 vaccinations in various healthcare settings and geographic locations. Despite the different data sources, the overall results as well as many of subgroup results were consistent. To summarize the results from these 2 data sources, VE estimates from a meta-analysis are presented. Statistical heterogeneity was observed in many of these results, based on the I2 statistic, which in some settings would prompt reconsideration of the appropriateness of meta-analysis. However, because our analytical approaches in each data source were identical and the populations similar, we did not find evidence of clinical heterogeneity and thus considered meta-analysis to be a reasonable approach to summarizing, smoothing, and drawing inferences.34,35
The eligibility and matching criteria were designed to avoid selection bias by identifying vaccinated and unvaccinated individuals who were eligible for vaccination on each calendar day. Starting follow-up on Time 0 without considering future vaccination behaviors avoided immortal person-time bias by not using future information to define exposure status at baseline.17 The daily VE estimates allowed for granular evaluation of changing VE over time.
Initial vaccination rollout focused on higher-risk individuals, but the exact prioritization procedures and groups varied by jurisdiction and changed over time. Because authorizations and recommendations for vaccination changed rapidly during the initial vaccine role out, defining time period- and geography-specific eligibility criteria for the entire study population was impractical; thus, we matched unvaccinated individuals to vaccinated on multiple indicators of eligibility status, including calendar date, immunocompromised status, pregnancy, comorbidities, and age; this matching approximated the rolling eligibility criteria by identifying unvaccinated individuals who were similar to vaccinated individuals on each calendar day in each US county of residence.
Despite the study’s matching on demographic and clinical characteristics and using propensity score weighting, residual confounding may remain. The generally increased VE estimates during the first 14 days of follow-up suggested a potential difference between the exposure groups during a time when vaccines are assumed to have no biologic effect while the body mounts an immune response to a novel antigen. However, other studies36 have demonstrated differential COVID-19 testing and diagnoses in the 3-4 days after Time 0, as recently vaccinated individuals may not seek COVID-19 testing, attributing symptoms to vaccine side effects. Additionally, diagnoses of COVID-19 recorded on the same day as vaccination may indicate recording of history of COVID-19,37 and these Time 0 events may be more common with vaccine brands administered in healthcare practice settings, rather than pharmacies or mass vaccination centers where concurrent diagnoses cannot be recorded. However, longer-term differences in healthcare-seeking behavior or residual unmeasured confounding cannot be ruled out.
COVID-19 testing capacity and capability have changed rapidly throughout the course of the pandemic, and COVID-19 laboratory testing results were not widely available in the databases utilized for this study, so COVID-19 diagnoses rather than confirmed COVID-19 infections were used as outcomes. COVID-19 severity is not accurately represented in diagnoses,38 so we evaluated COVID-19 diagnoses overall in any medically attended setting, and separately in hospital/ED settings as a proxy for severity. However, as the pandemic has progressed, COVID-19–related hospitalizations may be the most relevant public health measure for contemporary surveillance and prevention efforts.38,39 We matched vaccination groups on calendar time and evaluated outcomes over calendar time to account for the potentially changing meaning of a COVID-19 diagnosis over time (e.g., care seeking for COVID-19 versus incidental findings).
This real-world study is subject to many limitations common to observational research using existing data sources (e.g., key study elements may be misclassified or missing, the observed VEs may be subject to confounding by unmeasured characteristics, and these results may not be generalizable to other populations). We used health insurance claims supplemented with IIS encounters to identify vaccination status, substantially increasing vaccination capture compared to either data source alone.19 However, some vaccines may still be missed, resulting in differential misclassification of exposure. We expect that some truly vaccinated persons may have been classified as unvaccinated due to missing or unidentifiable records, recording errors, or other technical challenges that resulted in incomplete recording of information during the pandemic response. Given this potential for less than 100% sensitivity of our exposure classification records, VE may be underestimated; truly vaccinated individuals, along with their presumably lower risk of COVID-19, would be included in the unvaccinated pool, and thus decrease estimates of VE. To address this potential misclassification, we performed a QBA to determine how our estimates of VE would change based on hypothesized rates of underreporting of vaccination; for medically diagnosed COVID-19, the observed VE may underestimate the true VE by up to 14%. The extent of vaccination completeness and exposure misclassification was estimated using CDC and state-level estimates from the entire population aged less than 65 years, which may not always perfectly align with the study population. Thus, these bias analyses should be interpreted as suggestive of general estimates of direction and magnitude of bias, and not perfectly corrected estimates. The capture of covariates in health insurance diagnosis data may be affected by pandemic-related reductions in healthcare access and utilization, which also may change over the study period.
Receiving a complete primary series of BNT162b2, mRNA-1273, or JNJ-7836735 was associated with reductions in medically diagnosed COVID-19 incidence and hospital/ED-diagnosed COVID-19 in the US adult population, although the observed VE estimates differed across vaccine brands and by variant era. In the rapidly changing dynamics of the COVID-19 pandemic, additional real-world evidence is important for evaluating and understanding changes in VE and impacts of waning immunity on serious COVID-19 outcomes over time.
1. Financial support
This work was supported by the United States Food and Drug Administration as part of the BEST Initiative.
2. Data-availability statement
The data that support the findings of this study are available from Optum and CVS Health. Restrictions apply to the availability of these data which were authorized for use in this study.
3. The authors thank the following for their contributions
Sarah Harris, MA, of RTI International for project management and support; Dr. Melissa McPheeters, PhD, of RTI International for supervision; Kathryn Matuska, BA, and Wenxuan Zhou, BS, of Acumen LLC for editorial review of the manuscript and interpretation of results; Vivian Wilt, MPH, Julia McIlmail, BS, and Rebecca Warsawski, MPH, of Optum Epidemiology for Project management; Michael Kirksey, MBA, of Optum Enterprise Analytics for data analysis; Dr. Alexandra Stone, PhD, MSc, and Dr. Wafa Tarazi, PhD, MHPA, of Optum Serve Consulting for immunization data integration and quality assurance; Emily Myers, MPH, of Optum Serve Consulting for analytical support; Eli Wolter of Optum Technology for immunization data acquisition and processing; Edward Novak, MS, of Optum Technology for senior technology leadership; Megan Ketchell, BS, of Optum Serve Consulting for project coordination and communications support; Kathryn Federici, MSW, of Optum Serve Consulting for communications lead and legal liaison; Amaka Ume, MPH, of Optum Serve Consulting for analytical support.
4. Meeting presentation
Portions of this work have been accepted for a podium presentation at the 2024 ISPE Annual Meeting.
5. Disclaimer
This activity was conducted as part of the FDA public health surveillance mandate, and FDA did not require Institutional Review Board review; individual-level consent was not required for this analysis of secondary healthcare data.
6. Conflict of interest statement
JBL, CB, DI, MMR, and MSA are employees of RTI International, an independent, nonprofit research institute that performs research on behalf of governmental and commercial clients, including vaccine manufacturers. LP, JD, RO, RP, MM, JS, LBW, GWY, JS, and KA are employees of Optum and own stock in UnitedHealth Group. EJB is an employee of Optum. DAD and CNMW are employees of CVS Health. The other authors have no conflicts to report.
SUPPLEMENTAL INFORMATION
Supplemental Methods: Receiving Only 1 Dose of a 2-Dose Primary Series
This secondary analysis evaluated the effectiveness of receiving a single dose of a 2-dose primary series (BNT-162b2 or mRNA-1273), by vaccine brand, compared with being unvaccinated.
The individuals included in this analysis were drawn from the same brand-specific matched analytic cohorts used for the primary analysis (complete vaccine series versus being unvaccinated). Time 0 for both the vaccinated (receipt of Dose 1) and unvaccinated comparator (matched unvaccinated date) groups were the same, and the same 1:1 matching of vaccinated (with an eligible Dose 1 of BNT162b2 or mRNA-1273) and unvaccinated individuals was maintained. However, as this analysis evaluated a different vaccine exposure pattern than the primary analysis (i.e., receiving a single dose of a 2-dose primary series rather than receiving a complete primary series), the censoring criteria for deviation from the vaccine exposure pattern differed for the vaccinated group; vaccinated individuals were censored at the receipt of an additional COVID-19 vaccine dose after Dose 1, as follows:
The same propensity scores and IPT weights used in the primary analysis were used in this analysis. The IPT-weighted cumulative incidence curves of COVID-19 outcomes were plotted by exposure group, and HRs and 95% CIs with the corresponding VE were estimated using IPT-weighted Cox proportional hazards models.
Supplemental Methods: Quantitative Bias Analysis for Exposure Misclassification
Although vaccinations were identified in both health insurance claims data and IIS COVID-19 vaccine records [1], the possibility for missing vaccine information remains. Quantitative bias analyses [2,3] were performed to estimate the impact of truly vaccinated individuals being misclassified as unvaccinated due to missing vaccine records. No gold standard for vaccine status is available, so estimates of statewide receipt of at least 1 COVID-19 vaccine dose among individuals aged younger than 65 years from CDC, state departments of health, and capture-recapture methods [4-6] were obtained and compared with observed state-level estimates in the study data [1] to estimate maximum and minimum potential sensitivities of vaccine exposure.
Using the minimum and maximum sensitivity estimates from CDC, state departments of health, or capture-recapture methods, 2 “corrected” RR estimates (the primary analyses estimated HRs, but for the purposes of the quantitative bias analyses, RRs and 95% CIs were estimated in the weighted cohorts using a fixed 61-day follow-up time for both outcomes)—a minimum and maximum corrected estimate—were generated for each outcome by reassigning exposure status from unvaccinated to vaccinated based on the sensitivity estimate. A correction factor was then estimated as follows:
The bias correction factors were then applied to the observed HR estimates from the primary analyses. Specificity of the study’s vaccine assessment was assumed to be 100% (i.e., all observed claims or IIS records were assumed to be true vaccination events, and no truly unvaccinated individuals were misclassified as being vaccinated).
Footnotes
↵* Affiliation at time of study
Abbreviations
- CI
- confidence interval
- COVID-19
- coronavirus disease 2019
- ED
- emergency department
- HR
- hazard ratio
- IIS
- Immunization Information Systems
- Q1, Q3
- first and third quartiles
- QBA
- quantitative bias analysis
- RR
- risk ratio
- RVE
- relative vaccine effectiveness
- SD
- standard deviation
- sIPT
- stabilized inverse probability of treatment
- US
- United States
- VE
- vaccine effectiveness.