PT  - JOURNAL ARTICLE
AU  - Miller, April R.
AU  - Charepoo, Samin
AU  - Yan, Erik
AU  - Frost, Ryan W.
AU  - Sturgeon, Zachary J.
AU  - Gibbon, Grace
AU  - Balius, Patrick
AU  - Thomas, Cedonia S.
AU  - Schmitt, Melanie A.
AU  - Sass, Daniel A.
AU  - Walters, James B.
AU  - Flood, Tracy L.
AU  - Schmitt, Thomas A.
AU - on behalf of the COVID-19 Data Project
TI  - Reliability of COVID-19 data: An evaluation and reflection
AID  - 10.1101/2021.04.25.21256069
DP  - 2021 Jan 01
TA  - medRxiv
PG  - 2021.04.25.21256069
4099  - http://medrxiv.org/content/early/2021/04/27/2021.04.25.21256069.short
4100  - http://medrxiv.org/content/early/2021/04/27/2021.04.25.21256069.full
AB  - Importance The rapid proliferation of COVID-19 has left governments scrambling, and several data aggregators are now assisting in the reporting of county cases and deaths. The different variables affecting reporting (e.g., time delays in reporting) necessitates a well-documented reliability study examining the data methods and discussion of possible causes of differences between aggregators.Objective To statistically evaluate the reliability of COVID-19 across aggregators.Design, Setting, and Participants Cases and deaths were collected daily by volunteers via state and local health departments, as primary sources and newspaper reports, as secondary sources. In an effort to begin comparison for reliability statistical analysis, BroadStreet collected data from other COVID-19 aggregator sources, including USAFacts, Johns Hopkins University, New York Times, The COVID Tracking Project.Main Outcomes and Measures COVID-19 cases and death counts at the county and state levels.Results Lower levels of inter-rater agreement were observed across aggregators associated with the number of deaths, which manifested itself in state level Bayesian estimates of COVID-19 fatality rates.Conclusions and Relevance A national, publically available data set is needed for current and future disease outbreaks and improved reliability in reporting.Competing Interest StatementThe authors have declared no competing interest.Funding StatementNo external funding was received for this work.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:Data is secondary publicly available data distributed for public use and analysis by BroadStreet Health USAFacts, Johns Hopkins University, New York Times, The COVID Tracking ProjectAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesData is available at BroadStreet Health Github and includes county-level positive and probable COVID cases and deaths, and QA of all data points. https://github.com/BroadStreet-Health https://usafacts.org/visualizations/coronavirus-covid-19-spread-map/ https://github.com/CSSEGISandData/COVID-19 https://github.com/nytimes/covid-19-data https://github.com/COVID19Tracking