RT Journal Article
SR Electronic
T1 Ensemble Forecasts of Coronavirus Disease 2019 (COVID-19) in the U.S.
JF medRxiv
FD Cold Spring Harbor Laboratory Press
SP 2020.08.19.20177493
DO 10.1101/2020.08.19.20177493
A1 Ray, Evan L
A1 Wattanachit, Nutcha
A1 Niemi, Jarad
A1 Kanji, Abdul Hannan
A1 House, Katie
A1 Cramer, Estee Y
A1 Bracher, Johannes
A1 Zheng, Andrew
A1 Yamana, Teresa K
A1 Xiong, Xinyue
A1 Woody, Spencer
A1 Wang, Yuanjia
A1 Wang, Lily
A1 Walraven, Robert L
A1 Tomar, Vishal
A1 Sherratt, Katharine
A1 Sheldon, Daniel
A1 Reiner, Robert C
A1 Prakash, B. Aditya
A1 Osthus, Dave
A1 Li, Michael Lingzhi
A1 Lee, Elizabeth C
A1 Koyluoglu, Ugur
A1 Keskinocak, Pinar
A1 Gu, Youyang
A1 Gu, Quanquan
A1 George, Glover E.
A1 España, Guido
A1 Corsetti, Sabrina
A1 Chhatwal, Jagpreet
A1 Cavany, Sean
A1 Biegel, Hannah
A1 Ben-Nun, Michal
A1 Walker, Jo
A1 Slayton, Rachel
A1 Lopez, Velma
A1 Biggerstaff, Matthew
A1 Johansson, Michael A
A1 Reich, Nicholas G
A1 ,
YR 2020
UL http://medrxiv.org/content/early/2020/08/22/2020.08.19.20177493.abstract
AB Background The COVID-19 pandemic has driven demand for forecasts to guide policy and planning. Previous research has suggested that combining forecasts from multiple models into a single “ensemble” forecast can increase the robustness of forecasts. Here we evaluate the real-time application of an open, collaborative ensemble to forecast deaths attributable to COVID-19 in the U.S.Methods Beginning on April 13, 2020, we collected and combined one- to four-week ahead forecasts of cumulative deaths for U.S. jurisdictions in standardized, probabilistic formats to generate real-time, publicly available ensemble forecasts. We evaluated the point prediction accuracy and calibration of these forecasts compared to reported deaths.Results Analysis of 2,512 ensemble forecasts made April 27 to July 20 with outcomes observed in the weeks ending May 23 through July 25, 2020 revealed precise short-term forecasts, with accuracy deteriorating at longer prediction horizons of up to four weeks. At all prediction horizons, the prediction intervals were well calibrated with 92-96% of observations falling within the rounded 95% prediction intervals.Conclusions This analysis demonstrates that real-time, publicly available ensemble forecasts issued in April-July 2020 provided robust short-term predictions of reported COVID-19 deaths in the United States. With the ongoing need for forecasts of impacts and resource needs for the COVID-19 response, the results underscore the importance of combining multiple probabilistic models and assessing forecast skill at different prediction horizons. Careful development, assessment, and communication of ensemble forecasts can provide reliable insight to public health decision makers.Competing Interest StatementThe authors have declared no competing interest.Funding StatementELR, NGR, EC and others were supported by the US Centers for Disease Control and Prevention (U01IP001122). NGR, NW, and others were supported by the National Institutes for General Medical Sciences (R35GM119582). JB was supported by the Helmholtz Foundation via the SIMCARD Information & Data Science Pilot Project. The content is solely the responsibility of the authors and does not necessarily represent the official views of CDC or NIGMS. BAP and others were partially supported by the National Science Foundation (Expeditions CCF-1918770, CAREER IIS-1750407, RAPID IIS-2027862, Medium IIS-1955883, NRT DGE-1545362), funds from Georgia Tech Research Institute (GTRI) and funds/computing resources from Georgia Tech. The content is solely the responsibility of the authors and does not necessarily represent the views of the funding agencies, Georgia Tech, GTRI, University of Iowa, UIUC or IQVIA. SC and others, as well as computing resources, were funded by the University of Michigan. The content is solely the responsibility of the authors and does not necessarily represent the views of the University of Michigan. PK and others were supported in part by the William W. George and the Virginia C. and Joseph C. Mello endowments at Georgia Tech, the NSF grant MRI 1828187, and research cyberinfrastructure resources and services provided by the Partnership for an Advanced Computing Environment (PACE) at Georgia Tech. ECL and others were supported by the State of California, the U.S. Department of Health and Human Services and the U.S. Department of Homeland Security. This work was additionally supported by the Office of the Dean at the Johns Hopkins Bloomberg School of Public Health, the Johns Hopkins Health System, and with computing service credits from Amazon Web Services. These thoughts and opinions are our own and do not represent the views of the U.S. federal government. YW and others were supported by NIH grant GM124104. MBN and others were supported by the Defense Threat Reduction Agency (Award Number: HDTRA1-19-D-0007) and by the National Science Foundation (Award Number: 2031536). KS and others were supported by the Wellcome Trust [210758]. JC and others were supported by National Science Foundation awards 2035360 and 2035361. GE and others were supported by NSF RAPID award 2027718. RCR and others were supported by the Bill & Melinda Gates Foundation and National Science Foundation award 2031096. GEG and others were supported by funding provided by the US Army Corps of Engineers Geospatial Task Force. Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:UMass-Amherst IRB.All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesAll data and code referred to in the manuscript are publicly available. https://github.com/reichlab/covid19-forecast-hub/ https://github.com/reichlab/covidEnsembles https://zoltardata.com/project/44