Abstract
During the COVID-19 pandemic, many communities across the US experienced surges in hospitalizations, which strained the local hospital capacity and affected the overall quality of care. Even when effective vaccines became available, many communities remained at high risk of surges in COVID-19-related hospitalizations due to waning immunity, low uptake of booster vaccinations, and the continual emergence of new variations of SARS-CoV-2. Some risk metrics, such as the CDC’s Community Levels, were developed to predict the impact of COVID-19 on the community-level healthcare system based on routine surveillance data. However, they had limited utility as they were not routinely updated based on accumulating data and were not directly linked to specific outcomes, such as surges in COVID-19 hospitalizations beyond local capacities. Regression models could resolve these limitations, but they have limited interpretability and do not convey the reasoning behind their predictions. In this paper, we evaluated decision tree classifiers that were developed in “real-time” to predict surges in local hospitalizations due to COVID-19 between July 2020 and November 2022. These classifiers would have provided visually intuitive and interpretable decision rules for local decision-makers to understand and act upon, and by being updated weekly, would have responded to changes in the epidemic. We showed that these classifiers exhibit reasonable predictive ability with the area under the receiver operating characteristic curve (auROC) > 80%. These classifiers maintained their performance temporally (i.e, over the duration of the pandemic) and spatially (i.e., across US counties). We also showed that these classifiers outperformed the CDC’s Community Levels for predicting high hospital occupancy.
Significance Statement A major concern during the COVID-19 pandemic was the risk of exceeding local healthcare capacity due to COVID-19-related hospitalizations. To assess this risk and inform mitigating strategies, several risk assessment tools were developed during the pandemic. Many of these tools, however, did not predict local outcomes, were not updated as the pandemic progressed, and/or were not interpretable by decision-makers. We propose an adaptive framework of decision tree classifiers to predict whether COVID-19-related hospital occupancy would exceed a given capacity threshold. This framework would provide interpretable classification rules to predict surges in local hospitalizations,and maintained its performance over time and across US counties, and outperformed the CDC’s Community Level tool.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
Research reported in this publication was supported by the National Institute Of Allergy And Infectious Diseases of the National Institutes of Health under Award Number R21AI173746. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present work are contained in the manuscript and available online at https://github.com/nytimes/covid-19-data and https://healthdata.gov/Hospital/COVID-19-Reported-Patient-Impact-and-Hospital-Capa/anag-cw7u/about_data
