Effectiveness of mRNA COVID-19 vaccines and hybrid immunity in preventing SARS-CoV-2 infection and symptomatic COVID-19 among adults in the United States

Study population

We conducted an analysis on data spanning September 1, 2022 – July 31, 2023 from four sites in the United States to estimate COVID-19 vaccine effectiveness (VE) among adults aged ≥18 years. Specifically, we combined data from three prospective cohort studies, CASCADIA, CoVE (Community Vaccine Effectiveness against Asymptomatic and Symptomatic SARS-CoV-2 Infection in Michigan), and VIEW (Viruses and Infections in Essential Workers) [18, 19]. CASCADIA enrolled Kaiser Permanente Northwest and University of Washington patients and community members in the Portland, Oregon and Seattle, Washington metropolitan areas (both children and adults aged 18-49). CoVE enrolled children and adults of all ages who live in Michigan and receive healthcare and VIEW enrolled adults who are essential (non-healthcare) workers in Tennessee.

For this study, adults living in Washington, Oregon, Michigan, and Tennessee, including individuals from the same household, were eligible for inclusion. Written informed consent was obtained from all participants. This study was reviewed by the US Centers for Disease Control and Prevention and approved by the institutional review boards at participating sites and was conducted consistent with applicable federal law and CDC policy [20].

Data and specimen collection

At enrollment, participants completed an enrollment survey that included demographics, household characteristics, chronic medical conditions, COVID-19 vaccination history, and prior SARS-CoV-2 infection history; participants were resurveyed at regular intervals to capture up-to-date demographic information. Blood specimens were collected near to the time of enrollment from participants who consented to phlebotomy or by self-collection of blood specimens using Mitra or Tasso+ devices [21]. Weekly surveillance was conducted for COVID-like illness symptoms. Participants were asked to self-collect nasal swabs weekly, irrespective of symptoms. To optimally capture symptomatic COVID-19, participants were instructed to collect an additional respiratory specimen upon onset of symptoms if they occurred outside of the timing of their regular weekly swab cadence.

Laboratory Testing

All respiratory specimens were tested for SARS-CoV-2 using real-time reverse transcription-polymerase chain reaction (RT-PCR) based assays, as detailed in the supplementary materials (eTable 1). Of note, while the component studies (CASCADIA, CoVE, VIEW) do not fall under the medicolegal auspices of clinical testing, all three utilized the same molecular assays employed for patient care. Less than 1% of specimens generated PCR results that would be consider either ‘inconclusive’ or ‘failed’ under clinically validated parameters; for the purposes of the present analyses, these specimens were considered negative. Whole genome sequencing was attempted on all SARS-CoV-2-positive specimens with an adequate viral load [22-25].

Available serum specimens were tested for the presence of anti-nucleocapsid (N) IgG using a quantitative MesoScale Discovery (MSD) VPLEX assays (eTable 1). For the SARS-CoV-2 MSD assay, titers against the N protein were interpolated from a standard calibration curve provided by the manufacturer. Specimens below the lower limit of quantitation per assay insert were set to a value of half the lower limit. Per the assay insert, specimens were determined to have detectable anti-N IgG if they had a titer equal to or greater than 5,000 assigned units per mL (AU/mL).

Variables of interest

COVID-19 vaccination status was captured from enrollment and weekly/monthly surveys (self-report), vaccine cards provided by the participant, and/or from queries of the state immunization information systems and electronic medical records (EMR), when available. Vaccination data included vaccination dates, number of doses, and manufacturer. Information from the EMR and state immunization information systems was used preferentially over self-reported information in the event that a participant did not report a history of vaccination.

Symptomatic COVID-19 was defined as those with a positive RT-PCR test and at least two COVID-like illness symptoms reported within seven days before or after the specimen collection date. The list of COVID-like illness symptoms varied by the cohort study (eTable 2).

Prior infection was defined as laboratory-confirmation of infection by RT-PCR from a study-collected specimen prior to the analytic period, positive anti-N SARS-CoV-2 antibody at enrollment, or self-report of infection prior to enrollment or September 1, 2022 (whichever occurred later). Time since prior infection was categorized as no prior infection, <4 months, 4 – <6 months, 6 – <12 months, ≥12 months. Dates of prior infection were imputed for 198 (5.9%) participants who only had serologic results and therefore did not have dates associated with prior infection. Imputation was done using results from linear regression models, in which baseline nucleocapsid blood draw date and numeric nucleocapsid values served as the predictors for the date of prior infection (among study participants with known prior infection dates) (eMethods).

Statistical analysis

Descriptive statistics, comparing participants who became infected during the study period to participants who remained uninfected, included frequencies (proportions) for categorical variables and medians (interquartile ranges [IQRs]) for continuous variables. P-values were calculated using chi-square tests for categorical variables and Wilcoxon rank sum tests for continuous variables. The Andersen-Gill extension of the Cox proportional hazards model with time-varying vaccination status was used to estimate hazard ratios (HR) of first SARS-CoV-2 infections, comparing participants with receipt of a bivalent dose to participants who were either unvaccinated or had received the original monovalent vaccine ≥180 days prior [26]. Separate VE estimates were produced for SARS-CoV-2 infection (inclusive of asymptomatic and symptomatic infections), symptomatic COVID-19, vaccine valency (original monovalent vs. bivalent), and timing of vaccine receipt. VE estimates were stratified by prior infection status and variant period among those with recent bivalent vaccination (within 7-179 days). Additionally, VE estimates were produced for those with both vaccination and prior infection, with naïve (no evidence of prior infection) participants who were either unvaccinated or had received the original monovalent vaccine ≥180 days prior as the reference group.

Multivariable models adjusted for age, sex, race/ethnicity, presence of at least one underlying health conditions, time since prior infection, geographic site, household size, and 7-day average of COVID-19 cases per 100 000 by site (local incidence: modeled as a continuous linear variable). COVID-19 vaccine effectiveness (VE) was calculated as VE = (1 - HR) × 100. Confidence intervals were calculated using the standard estimation methods for the Cox proportional hazards model, because the cluster size of participants by household was small [27]. In particular, 35.4% of households had two or more adults included in the analysis (note: 89 participants had a missing household ID and were assumed to be the only study participant in the household).

Person-time was calculated as the total number of days under surveillance for a given vaccination status during the analytic period. The surveillance period started on September 1, 2022 and ended on the date of a participant’s first positive RT-PCR test, the participant’s study withdrawal date, or end of the analytic period (July 31, 2023), whichever came first. Individuals enrolled after September 1, 2022 began time at risk at the time of surveillance start or six weeks after prior infection, if recently infected prior to enrollment. The surveillance weeks for which there was no specimen result (e.g., participant skipped a weekly swab) for four or more consecutive weeks were censored. The two weeks following an original monovalent primary vaccine dose and the week following bivalent or original monovalent booster vaccine doses were also excluded from person-time. A sensitivity analysis was conducted by restricting the analysis period to November 27, 2022 – July 31, 2023 in order to account for the difference in enrollment start dates of the studies (e.g. CASCADIA and CoVE began enrollment in July and August of 2022, whereas VIEW started in November 2022) and for the differences in the percent of participants who received a bivalent vaccine by site (CASCADIA=68.2%, CoVE=59.9%, and VIEW=25.4%).

All analyses were conducted using SAS software (version 9.4; SAS Institute, Cary, NC) or R Studio software (version 4.1.0; R Foundation).

Study population

Between September 1, 2022 and July 31, 2023, 3,343 participants contributed to person-time in the analysis, 50.0% from CASCADIA, 39.4% from VIEW, and 10.6% from CoVE (Table 1). Overall, 67.2% were female, median age was 41 years (interquartile range [IQR]: 36-46 years), and the majority were White, non-Hispanic (69.2%). Almost half of participants lived in a household with 4 or more individuals (48.1%), and 60.2% of participants reported having at least one chronic health condition. Age, gender, race/ethnicity, prevalence of chronic conditions, and household size varied by site (Table e3). Of the 1462 prior infections reported, 13.3% were from self-report only. During the study period, 21.7% (n=727) of participants had a laboratory-confirmed SARS-CoV-2 infection. A higher proportion of participants living in a household with more than one other person had SARS-CoV-2 infections than those who lived alone (20.6% versus 14.2%). A higher proportion of those with no documented prior infection had SARS-CoV-2 infection during the study period than those with a prior infection (26.2% versus 16.1%). Among participants with SARS-CoV-2 infections during the study period, 62.3% reported symptomatic COVID-19. Of the 727 SARS-CoV-2 infections, 400 (55%) had genetic sequencing results; the most prevalent lineages were XBB (60.3%), BQ.1.1 (14.8%), and BA.4/BA.5 (14.0%).

Vaccine uptake

Half of participants received at least one bivalent COVID-19 vaccine dose (49.7%) (Table 2). Participants enrolled from the Kaiser Permanent Northwest health plan (Oregon and Washington) had the highest uptake of bivalent vaccine doses (77.0%), whereas those in Tennessee (VIEW) had the lowest (25.4%). Black, non-Hispanic participants had the lowest reported proportion of receiving bivalent vaccine (27.8%), followed by Hispanic participants (34.5%), compared to White, non-Hispanic participants (54.1%). Those without report of a prior infection had higher uptake of bivalent vaccine (53.4%) compared to those with report of a prior infection (44.9%).

Vaccine effectiveness against infection

Of the 727 SARS-CoV-2 infections, 406 (55.8%) were among participants who were either unvaccinated/received a monovalent vaccine dose ≥180 days prior (1.74 infections per 1,000 person-days, [95% CI: (1.56–1.93]), and 321 (44.2%) were among those who received a bivalent dose (1.20 infections per 1,000 person-days, [95% CI: (1.08–1.34]) (Table 2 and 3). Adjusted VE of a bivalent dose received within 7-60 days against laboratory-confirmed SARS-CoV-2 infection, compared to the reference of being unvaccinated/receiving an original monovalent vaccine dose ≥180 days prior was 37.2% (95% CI: 11.4–58.5%) (Table 3). Compared to the same reference, adjusted VE of a bivalent dose received within 60-179 days was 17.0% (95% CI: -3.7-33.2%), and adjusted VE of a bivalent dose received ≥180 days prior was 8.7% (95% CI: -16.3-30.6%). When stratified by prior infection status, adjusted VE of a bivalent dose within 7-179 days against infection was 26.5% (95% CI: -1.1-40.8) among those who were naïve and 37.7% (95% CI: 6.8-58.2%) among those with prior infection. Adjusted VE of the original monovalent vaccine within 180 days against laboratory-confirmed SARS-CoV-2 infection, compared to the same reference group, was 26.7% (95% CI: -0.7-44.0%).

Hybrid immunity against infection and symptomatic illness

The combined protection from bivalent vaccination and prior infection compared to naïve participants who were unvaccinated/received a monovalent vaccine dose ≥180 days prior was 62.2 (95% CI: 44.2-74.6%) when vaccination was received within 7-179 days and 39.4% (95% CI: 11.7-61.3%) when received ≥180 days prior (Table 4). For symptomatic COVID-19 illness, combined protection was 73.0% (95% CI: 57.5-83.9%) when bivalent vaccination was received within 7-179 days and 56.7% (95% CI: 29.7-77.3%) when received ≥180 days prior.

The combined protection from monovalent vaccination within 7-179 days and prior infection compared to the same referenced group was 59.5% (95% CI: 31.5-80.4%) against laboratory-confirmed SARS-CoV-2 infection and 80.6% (95% CI: 59.2-94.1%) against symptomatic COVID-19 illness.

Sensitivity analysis

In a sensitivity analysis, limiting the analysis timeframe to November 27, 2022 – July 31, 2023, adjusted VE of bivalent dose received within 7-60 days against infection was 46.2% (95% CI: 17.5–66.3%), received within 7-180 days was 26.9% (95% CI: 9.6–40.6%), and received ≥180 days was 8.5% (95% CI: - 19.0–31.8%) (Table e4). Overall, adjusted VE of a bivalent dose, regardless of timing of vaccine receipt, was 19.9% (95% CI: 3.2–33.2%). When examining protection from both vaccination and prior infection, protection against infection and symptomatic COVID-19 was similar to the main analysis results (Table e5).

Limitations

There are several important limitations of this study. First, RT-PCR testing methods and COVID-like illness definitions varied by cohort site; therefore, some differences in definition of infection or symptomatic COVID-19 may be present. Second, weekly or symptomatic RT-PCR testing prior to the analytic study start date for estimation of prior infection history was only available among a subset of participants. To address this concern, we incorporated serologic data in order to identify additional prior SARS-CoV-2 infections but due to anti-N SARS-CoV-2 antibody waning, some prior infections may have been undetected. Third, social desirability or recall bias may have affected self-report of prior infection and vaccination status when RT-PCR and serologic test results and data from the state immunization information systems and EMR were unavailable. Fourth, vaccination may be associated with other protective factors that may be difficult to ascertain and account for fully. Fifth, limited sample sizes resulted in imprecise VE estimates and should be interpreted with caution, as the imprecision may indicate that the actual VE could be substantially different from the point estimates shown. Last, these observations derived from three large prospective cohort studies, while internally valid, may not directly generalize to other settings.

This study also has many strengths, including more than 3,300 participants enrolled from four distinct sites in the U.S. Participants swabbed weekly, regardless of symptoms, which greatly reducing the risk of missing an asymptomatic SARS-CoV-2 infection. Furthermore, adherence to weekly swabbing was high (median: 85%). Weekly and quarterly surveys, as well as data from the state immunization information systems and EMR, ensured detailed and complete information on potential confounding variables and vaccination status.