Risk of Coronavirus Disease 2019 (COVID-19) among Those Up-to-Date and Not Up-to-Date on COVID-19 Vaccination ============================================================================================================= * Nabin K. Shrestha * Patrick C. Burke * Amy S. Nowacki * Steven M. Gordon ## ABSTRACT **Background** The CDC recently defined being “up-to-date” on COVID-19 vaccination as having received at least one dose of a COVID-19 bivalent vaccine. The purpose of this study was to compare the risk of COVID-19 among those “up-to-date” and “not up-to-date” on COVID-19 vaccination. **Methods** Employees of Cleveland Clinic in employment when the COVID-19 bivalent vaccine first became available, and still employed when the XBB lineages became dominant, were included. Cumulative incidence of COVID-19 since the XBB lineages became dominant was compared across the “up-to-date” and “not up-to-date” states, by treating COVID-19 bivalent vaccination as a time-dependent covariate whose value changed on receipt of the vaccine. Risk of COVID-19 by vaccination status was also compared using multivariable Cox proportional hazards regression adjusting for propensity to get tested for COVID-19, age, sex, and phase of most recent prior SARS-CoV-2 infection. **Results** COVID-19 occurred in 1475 (3%) of 48 344 employees during the 100-day study period. The cumulative incidence of COVID-19 was lower in the “not up-to-date” than in the “up-to-date” state. On multivariable analysis, not being “up-to-date” with COVID-19 vaccination was associated with lower risk of COVID-19 (HR, 0.77; 95% C.I., 0.69-0.86; *P*-value, <0.001). Results were very similar when those 65 years and older were only considered “up-to-date” after receiving 2 doses of the bivalent vaccine. **Conclusions** Since the XBB lineages became dominant, adults “not up-to-date” by the CDC definition have a lower risk of COVID-19 than those “up-to-date” on COVID-19 vaccination, bringing into question the value of this risk classification definition. **Summary** Among 48 344 working-aged Cleveland Clinic employees, those not “up-to-date” on COVID-19 vaccination had a lower risk of COVID-19 than those “up-to-date”. The current CDC definition provides a meaningless classification of risk of COVID-19 in the adult population. Keywords * SARS-CoV-2 * COVID-19 * effectiveness * vaccines * bivalent vaccine ## INTRODUCTION In April 2023 the Centers for Disease Control and Prevention (CDC) updated its guidance on Coronavirus Disease 2019 (COVID-19) vaccination to consider all individuals above the age of 6 to be “up-to-date” with COVID-19 vaccination only if they had received at least one dose of a COVID-19 bivalent vaccine [1]. By this definition, those who had not received a single dose of a COVID-19 bivalent vaccine would be considered not “up-to-date”. A recent study was unable to find the bivalent vaccine to be effective while the XBB variants were the dominant circulating strains [2]. Given this lack of effectiveness of the bivalent vaccine against the XBB variants, which are the predominant circulating strains at this time, it is reasonable to question whether individuals “up-to-date” with a vaccine of questionable effectiveness are protected against COVID-19 compared to those not “up-to-date”. The purpose of this study was to evaluate whether individuals who were not “up-to-date” on COVID-19 vaccination had a higher risk of COVID-19 than those “up-to-date”. ## METHODS ### Study Design This was a retrospective cohort study conducted at the Cleveland Clinic Health System (CCHS) in the United States. ### Patient Consent Statement The study was approved by the Cleveland Clinic Institutional Review Board as exempt research (IRB no. 22-917). A waiver of informed consent and waiver of HIPAA authorization were approved to allow the research team access to the required data. ### Setting Since the arrival of the COVID-19 pandemic at Cleveland Clinic in March 2020, employee access to testing has been a priority. Voluntary vaccination for COVID-19 began on 16 December 2020, and the monovalent mRNA vaccine as a booster became available to employees on 5 October 2021. The COVID-19 bivalent mRNA vaccine began to be offered to employees on 12 September 2022. The study start date was 29 January 2023, the date the XBB lineages first became the dominant circulating strains in Ohio. ### Participants CCHS employees in employment at any location in Ohio when the COVID-19 bivalent vaccine first became available (12 September 2022), and who were still employed when the XBB lineages became dominant (29 January 2023), were included in the study. Those for whom age and sex were not available were excluded. ### Variables Covariates collected were age, sex, job location, and prior SARS-CoV-2 infection, as described in our earlier studies [2–5]. Institutional data governance rules related to employee data limited our ability to supplement our dataset with additional clinical variables. Vaccination status was “up-to-date” or “not up-to-date” based on whether or not a person had received at least one dose of a COVID-19 bivalent vaccine. For each subject, the propensity to get tested for COVID-19 was defined as the number of COVID-19 nucleic acid amplification tests (NAATs) done divided by the number of years of employment at CCHS during the pandemic. The pandemic phase during which a subject had his or her last prior episode of COVID-19 was also collected as a variable, based on which variant/lineages accounted for more than 50% of infections in Ohio at the time [6]. ### Outcome The study outcome was time to COVID-19, the latter defined as a positive NAAT for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) any time after the study start date. Outcomes were followed until 10 May 2023, allowing for evaluation of outcomes up to 100 days from the study start date. ### Statistical Analysis A Simon-Makuch hazard plot [7] was created to compare the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states, by treating vaccination status as a time-dependent covariate. A subject’s vaccination status was “not up-to-date” before receipt of a COVID-19 bivalent vaccine, and “up-to-date” after receipt of the vaccine. Subjects whose employment was terminated during the study period before they had COVID-19 were censored on the date of termination. Curves for the “not up-to-date” state were based on data while the vaccination status was “not up-to-date”. Curves for the “up-to-date” state were based on data from the date the vaccination status changed to “up-to-date”. Multivariable Cox proportional hazards regression models were fit to examine the association of various variables with time to COVID-19. Vaccination status was included as a time-dependent covariate [8]. The possibility of multicollinearity in the models was evaluated using variance inflation factors. The proportional hazards assumption was checked using log(-log(survival)) vs. time plots. The hazard ratio for vaccination status provided a comparison of risk of COVID-19 based on vaccination status after adjusting for all other covariates. The analysis was performed by N. K. S. and A.S.N. using the *survival* package and R version 4.2.2 (R Foundation for Statistical Computing) [8–10]. ## RESULTS Of 48 344 included subjects, 1445 (3%) were censored during the study because of termination of employment. By the end of the study, 12 841(27%) were “up-to-date” on COVID-19 vaccination, of whom 11 187 (87%) received the Pfizer vaccine and 1654 (13%) received the Moderna vaccine. A total of 1475 employees (3%) acquired COVID-19 during the 100 days of the study. ### Baseline Characteristics Table 1 shows the characteristics of subjects included in the study. Notably, this was a relatively young population, with a mean age of 43 years. Among these, 22 407 (46%) had previously had a documented episode of COVID-19 and 16 262 (34%) had previously had an Omicron variant infection. 42 160 subjects (87%) had previously received at least one dose of vaccine and 44 432 (92%) had been previously exposed to SARS-CoV-2 by infection or vaccination. Altogether, 36 490 subjects (76%) were tested for COVID-19 by a NAAT at least once while employed at Cleveland Clinic. The propensity for COVID-19 testing ranged from 0 to 63.5 per year, with a median of 0.64 and interquartile range spanning 0.32 to 1.27 per year. View this table: [Table 1.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/T1) Table 1. Baseline Characteristics of 48 344 Employees of Cleveland Clinic in Ohio ### Risk of COVID-19 Based on Vaccination Status and Prior Infection The risk of COVID-19 was lower in the “not up-to-date” state than in the “up-to-date” state, with respect to COVID-19 vaccination (Figure 1). When stratified by tertiles of propensity to get tested for COVID-19, the “not up-to-date” state was not associated with a higher risk of COVID-19 than the “up-to-date” state in any tertile (Figure 2). ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/06/12/2023.06.09.23290893/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/F1) Figure 1. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states with respect to COVID-19 vaccination. Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/06/12/2023.06.09.23290893/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/F2) Figure 2. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states with respect to COVID-19 vaccination, stratified by tertiles of propensity to get tested for COVID-19. Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. Solid lines represent the “up-to-date” states while dashed lines represent the “not up-to-date” states. In contrast, consideration of prior infection provided a more accurate classification for risk of COVID-19. Those whose last prior SARS-CoV-2 infection occurred during the Omicron BQ or BA.4/BA.5 dominant phases had a substantially and significantly lower risk of COVID-19 than those whose last SARS-CoV-2 infection occurred in the early Omicron or pre-Omicron periods or those not previously known to have had COVID-19 (Figure 3). When stratified by most recent prior SARS-CoV-2 infection, there was no difference in risk of COVID-19 between the “up-to-date” and “not up-to-date” states within each most recent prior infection group, except for those not previously known to be infected, among whom the “up-to-date” state was associated with a higher risk of COVID-19 than the “not up-to-date” state (Figure 4). ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/06/12/2023.06.09.23290893/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/F3) Figure 3. Hazard plot comparing the cumulative incidence of COVID-19 stratified by the phase during which the last prior episode of COVID-19 occurred. Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. Solid lines represent those last infected before the Omicron variants became dominant or those not previously known to have had COVID-19, while dashed lines represent those last infected while the Omicron variants were dominant. ![Figure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/06/12/2023.06.09.23290893/F4.medium.gif) [Figure 4.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/F4) Figure 4. Simon-Makuch hazard plot comparing the cumulative incidence of COVID-19 in the “up-to-date” and “not up-to-date” states with respect to COVID-19 vaccination, stratified by the phase during which the last prior episode of COVID-19 occurred. Day zero was 29 January 2023, the day the XBB lineages of the Omicron variant became the dominant strains in Ohio. Point estimates and 95% confidence intervals are jittered along the x-axis to improve visibility. “Remote infection” includes infections during the pre-Omicron and Omicron BA.1/BA.2 dominant periods, i.e more than 218 days before the study start date. Solid lines represent the “up-to-date” states while dashed lines represent the “not up-to-date” states. ### Multivariable Analysis In a multivariable Cox proportional hazards regression model, adjusted for propensity to get tested for COVID-19, age, sex, and phase of most recent SARS-CoV-2 infection, not being “up-to-date” on COVID-19 vaccination was associated with a lower risk of COVID-19 (HR, 0.77; 95% C.I., 0.69-0.86; *P*-value, <0.001). Having had their last prior episode of COVID-19 while lineages of the Omicron variant were dominant was protective against COVID-19 with higher protection the more recent the infection. Point estimates of hazard ratios and associated 95% confidence intervals for the variables included in the unadjusted and adjusted Cox proportional hazards regression models are shown in Table 2. Results were very similar if those 65 years and older were only considered “up-to-date” after receipt of at least 2 doses of a bivalent vaccine (Supplementary Table 1). Results were also very similar if an individual was only considered “up-to-date” 7 days after receipt of a bivalent vaccine (Supplementary Table 2). View this table: [Table 2.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/T2) Table 2. Unadjusted and Adjusted Associations With Time to COVID-19 ## DISCUSSION This study found that not being “up-to-date” on COVID-19 vaccination, using the current CDC definition, was associated with a lower risk of COVID-19 than being “up-to-date”, while the XBB lineages were the dominant circulating strains of SARS-CoV-2. There are two reasons why not being “up-to-date” on COVID-19 vaccination by the CDC definition was associated with a lower risk of COVID-19. The first is that the bivalent vaccine was somewhat effective against strains that were more similar to the strains on the basis of which the bivalent vaccine was developed, but is not effective against the XBB lineages of the Omicron variant [2]. The second is that the CDC definition does not consider the protective effect of immunity acquired from prior infection. Because the COVID-19 bivalent vaccine provided some protection against the BA.4/BA.5 and BQ lineages [2], those “not-up-to-date” were more likely than those “up-to-date” to have acquired a BA.4/BA.5 or BQ lineage infection when those lineages were the dominant circulating strains. It is now well-known that SARS-CoV-2 infection provides more robust protection than vaccination [4,11,12]. Therefore it is not surprising that not being “up-to-date” according to the CDC definition was associated with a higher risk of prior BA.4/BA.5 or BQ lineage infection, and therefore a lower risk of COVID-19, than being “up-to-date”, while the XBB lineages were dominant. The strengths of our study include its large sample size, and its conduct in a healthcare system that devoted resources to have an accurate accounting of who had COVID-19, when COVID-19 was diagnosed, who received a COVID-19 vaccine, and when. The study methodology, treating vaccination status as a time-dependent covariate, allowed for determining vaccine effectiveness in real time. Adjusting for the propensity to get tested for COVID-19 should have mitigated against concern that individuals who bothered to remain up-to-date on COVID-19 vaccination may have been more likely to get tested for COVID-19 when they had symptoms. Our study was limited to examination of all detected infections. We were unable to distinguish between symptomatic and asymptomatic infections, and the rarity of severe illnesses precluded studying whether being “up-to-date” decreased severity of illness. Information on prior COVID-19 would have been incomplete, as many asymptomatic and mildly symptomatic infections would have been missed. There may have been unconsidered variables that might have influenced the results. Lastly, our study was done in a healthcare population, and included no children and few elderly subjects, and the majority of study subjects would not have been immunocompromised. This study’s findings question the wisdom of promoting the idea that every person needs to be “up-to-date” on COVID-19 vaccination, as currently defined, at this time. It is often stated that the primary purpose of vaccination is to prevent severe COVID-19 and death. We certainly agree with this, but it should be pointed out that there is not a single study that has shown that the COVID-19 bivalent vaccine protects against severe disease or death caused by the XBB lineages of the Omicron variant. At least one prior study has failed to find a protective effect of the bivalent vaccine against the XBB lineages of SARS-CoV-2 [2]. People may still choose to get the vaccine, but an assumption that the vaccine protects against severe disease and death is not reason enough to unconditionally push a vaccine of questionable effectiveness to all adults. In conclusion, this study found that not being “up-to-date” on COVID-19 vaccination by the CDC definition was associated with a lower risk of COVID-19 than being “up-to-date”. This study highlights the challenges of counting on protection from a vaccine when the effectiveness of the vaccine decreases over time as new variants emerge that are antigenically very different from those used to develop the vaccine. It also demonstrates the folly of risk classification based solely on receipt of a vaccine of questionable effectiveness while ignoring protection provided by prior infection. ## Notes ### Author contributions N. K. S.: Conceptualization, methodology, validation, investigation, data curation, software, formal analysis, visualization, writing-original draft preparation, writing-reviewing and editing, supervision, project administration. P. C. B.: Resources, investigation, validation, writing-reviewing and editing. A. S. N.: Methodology, formal analysis, visualization, validation, writing-reviewing and editing. S. M. G.: Project administration, resources, writing-reviewing and editing. ### Potential conflicts of interest The authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed. ### Funding None. ## Data Availability All data produced in the present study are available upon reasonable request to the authors ## SUPPLEMENTARY TABLES View this table: [Supplementary Table 1.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/T3) Supplementary Table 1. Adjusted Associations With Time to COVID-19 When Those Above Age 65 Needed Two Vaccine Doses to Be Considered “Up-To-Date” View this table: [Supplementary Table 2.](http://medrxiv.org/content/early/2023/06/12/2023.06.09.23290893/T4) Supplementary Table 2. Adjusted Associations With Time to COVID-19, When a Person Was Considered “Up-To-Date” Only After 7 Days Had Passed Since Receipt of the Bivalent Vaccine * Received June 9, 2023. * Revision received June 9, 2023. * Accepted June 12, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## REFERENCES 1. 1.COVID-19 Vaccination. 2023. Available at: [https://www.cdc.gov/coronavirus/2019-ncov/vaccines/stay-up-to-date.html](https://www.cdc.gov/coronavirus/2019-ncov/vaccines/stay-up-to-date.html). Accessed 9 May 2023. 2. 2.Shrestha NK, Burke PC, Nowacki AS, Simon JF, Hagen A, Gordon SM. Effectiveness of the Coronavirus Disease 2019 (COVID-19) Bivalent Vaccine. Open Forum Infect Dis 2023; :ofad209. 3. 3.Shrestha NK, Nowacki AS, Burke PC, Terpeluk P, Gordon SM. Effectiveness of mRNA COVID-19 Vaccines among Employees in an American Healthcare System. 2021: 2021.06.02.21258231. Available at: [https://www.medrxiv.org/content/10.1101/2021.06.02.21258231v1](https://www.medrxiv.org/content/10.1101/2021.06.02.21258231v1). Accessed 10 August 2021. 4. 4.Shrestha NK, Burke PC, Nowacki AS, Terpeluk P, Gordon SM. Necessity of Coronavirus Disease 2019 (COVID-19) Vaccination in Persons Who Have Already Had COVID-19. Clin Infect Dis Off Publ Infect Dis Soc Am 2022; 75:e662–e671. 5. 5.Shrestha NK, Shrestha P, Burke PC, Nowacki AS, Terpeluk P, Gordon SM. Coronavirus Disease 2019 Vaccine Boosting in Previously Infected or Vaccinated Individuals (COVID-19). Clin Infect Dis 2022; 75:2169–2177. 6. 6.Lambrou AS. Genomic Surveillance for SARS-CoV-2 Variants: Predominance of the Delta (B.1.617.2) and Omicron (B.1.1.529) Variants — United States, June 2021–January 2022. MMWR Morb Mortal Wkly Rep 2022; 71. Available at: [https://www.cdc.gov/mmwr/volumes/71/wr/mm7106a4.htm](https://www.cdc.gov/mmwr/volumes/71/wr/mm7106a4.htm). Accessed 15 December 2022. 7. 7.Simon R, Makuch RW. A non-parametric graphical representation of the relationship between survival and the occurrence of an event: Application to responder versus non-responder bias. Stat Med 1984; 3:35–44. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.4780030106&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=6729287&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F06%2F12%2F2023.06.09.23290893.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1984SM15700005&link_type=ISI) 8. 8.Therneau TM, Crowson C, Atkinson E. Using time dependent covariates and time dependent coefficients in the Cox model. 2021; Available at: [https://cran.r-project.org/web/packages/survival/vignettes/timedep.pdf](https://cran.r-project.org/web/packages/survival/vignettes/timedep.pdf). Accessed 8 May 2021. 9. 9.Therneau TM, Grambsh PM. Modeling Survival Data: Extending the Cox Model. New York, NY: Springer International Publishing, 2000. 10. 10.R Core Team. R: A language and environment for statistical computing. 2022; 11. 11.Gazit S, Shlezinger R, Perez G, et al. Comparing SARS-CoV-2 natural immunity to vaccine-induced immunity: reinfections versus breakthrough infections. 2021: 2021.08.24.21262415. Available at: [https://www.medrxiv.org/content/10.1101/2021.08.24.21262415v1](https://www.medrxiv.org/content/10.1101/2021.08.24.21262415v1). Accessed 25 August 2021. 12. 12.León TM, Dorabawila V, Nelson L, et al. COVID-19 Cases and Hospitalizations by COVID-19 Vaccination Status and Previous COVID-19 Diagnosis — California and New York, May– November 2021. MMWR Morb Mortal Wkly Rep 2022; 71:125–131. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.15585/mmwr.mm7104e1&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F06%2F12%2F2023.06.09.23290893.atom)