Abstract
Background A large number of online COVID-19 symptom checkers and chatbots have been developed but anecdotal evidence suggests that their conclusions are highly variable. To our knowledge, no study has evaluated the accuracy of COVID-19 symptom checkers in a statistically rigorous manner.
Methods In this paper, we evaluate 10 different COVID-19 symptom checkers screening 50 COVID-19 case reports alongside 410 non-COVID-19 control cases.
Results We find that the number of correctly assessed cases varies considerably between different symptom checkers, with Symptoma (F1=0.92, MCC=0.85) showing the overall best performance followed by Infermedica (F1=0.80, MCC=0.61).
Competing Interest Statement
All authors are employees of Symptoma GmbH. JN holds shares of Symptoma.
Funding Statement
This study has received funding from the European Unions Horizon 2020 research and innovation programme under grant agreement No 830017
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
pure computer work on public data
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Paper in collection COVID-19 SARS-CoV-2 preprints from medRxiv and bioRxiv
The Chan Zuckerberg Initiative, Cold Spring Harbor Laboratory, the Sergey Brin Family Foundation, California Institute of Technology, Centre National de la Recherche Scientifique, Fred Hutchinson Cancer Center, Imperial College London, Massachusetts Institute of Technology, Stanford University, University of Washington, and Vrije Universiteit Amsterdam.