Optimal symptom combinations to aid COVID-19 case identification: analysis from a community-based, prospective, observational cohort ==================================================================================================================================== * M Antonelli * J Capdevila * A Chaudhari * J Granerod * LS Canas * MS Graham * K Klaser * M Modat * E Molteni * B Murray * CH Sudre * R Davies * A May * LH Nguyen * DA Drew * A Joshi * AT Chan * JP Cramer * T Spector * J Wolf * S Ourselin * CJ Steves * AE Loeliger ## Abstract **Objectives** Diagnostic work-up following any COVID-19 associated symptom will lead to extensive testing, potentially overwhelming laboratory capacity whilst primarily yielding negative results. We aimed to identify optimal symptom combinations to capture most cases using fewer tests with implications for COVID-19 vaccine developers across different resource settings and public health. **Methods** UK and US users of the COVID-19 Symptom Study app who reported new-onset symptoms and an RT-PCR test within seven days of symptom onset were included. Sensitivity, specificity, and number of RT-PCR tests needed to identify one case (test per case [TPC]) were calculated for different symptom combinations. A multi-objective evolutionary algorithm was applied to generate combinations with optimal trade-offs between sensitivity and specificity. **Findings** UK and US cohorts included 122,305 (1,202 positives) and 3,162 (79 positive) individuals. Within three days of symptom onset, the COVID-19 specific symptom combination (cough, dyspnoea, fever, anosmia/ageusia) identified 69% of cases requiring 47 TPC. The combination with highest sensitivity (fatigue, anosmia/ageusia, cough, diarrhoea, headache, sore throat) identified 96% cases requiring 96 TPC. **Interpretation** We confirmed the significance of COVID-19 specific symptoms for triggering RT-PCR and identified additional symptom combinations with optimal trade-offs between sensitivity and specificity that maximize case capture given different resource settings. **Highlights** * Widely recommended symptoms identified only ∼70% COVID-19 cases * Additional symptoms increased case finding to > 90% but tests needed doubled * Optimal symptom combinations maximise case capture considering available resources * Implications for COVID-19 vaccine efficacy trials and wider public health ## Introduction Safe and effective vaccines represent the most promising intervention to prevent morbidity and mortality during the coronavirus disease (COVID)-19 pandemic.1,2 Positive results have recently emerged from three ongoing vaccine efficacy trials of COVID-19 vaccines.3-5 However, further vaccines are required to meet global demand, and vaccines currently in early development may result in better tolerability profiles, scalability, impact on viral shedding, and may be suitable to specific population subgroups. Thus, further important COVID-19 vaccine efficacy trials are predicted to start soon. In a clinical trial, diagnostic testing of suspected cases (e.g., reverse transcription polymerase chain reaction [RT-PCR] for severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2]) could be triggered by the presence of any COVID-19 associated symptom. A household survey in the United Kingdom (UK) showed that fever, cough, anosmia, and ageusia were present on the day of testing in only 60% of symptomatic, RT-PCR positive individuals, implying that other less specific signs/symptoms associated with COVID-19 occur in a substantial number of patients.6 The signs/symptoms associated with COVID-19 are extensive and overlap with those of other common viral infections.7,8 Thus, diagnostic work-up following any COVID-19 associated symptom may lead to indiscriminate testing and potentially overwhelm laboratory capacity whilst primarily yielding negative results. Identification of an efficient symptom combination to trigger diagnostic work-up that will capture the majority of COVID-19 cases using the lowest possible number of tests would enable optimum use of laboratory and financial resources in future vaccine efficacy trials. This would also be of wider benefit in public health settings for the early detection of symptomatic SARS-CoV-2 infection. Such data are scant and the triggering symptoms vary between publicly available vaccine efficacy trial protocols.9-15 We simulate COVID-19 case finding in a trial population using a community-based, prospective, observational cohort study. Data from UK COVID Symptom Study app 16 users were used to quantify how much individual COVID-19 symptoms contribute to COVID-19 case finding and to generate symptom combinations with optimal trade-offs between sensitivity and specificity that maximise the capture of RT-PCR positive cases given different laboratory capacities. The findings were replicated in a dataset of COVID Symptom Study app users in the United States (US). ## Material ### Study design and data source A community-based cohort study was carried out using data from the COVID Symptom Study app, a free smartphone app launched at the end of March 2020 and developed by Zoe Global (London, UK) in collaboration with King’s College London (London, UK) and Massachusetts General Hospital (Boston, MA, USA).16 Users from UK and US report baseline demographic information, data on comorbidities and COVID-19 testing results, and are encouraged to self-report a set of pre-specified symptoms on a daily basis to enable collection of longitudinal information on incident symptoms. This study was approved by the Partners Human Research Committee (Protocol 2020P000909) and King’s College London ethics committee (REMAS ID 18210, LRS-19/20-18210). ### Study population Individuals were included in the study if they met the following criteria: 1) aged≥18 years, 2) reported developing any symptom between March 24th and September 15th, 2020, and 3) entered a valid RT-PCR test result within the first seven days of symptom onset. App users who recorded a history of COVID-19 were excluded. Data were frozen and extracted on October 21st, 2020. UK participants served as a discovery cohort, which was randomly split into training and validation datasets of equal size. US participants served as a replication cohort to confirm the generalisability of the results. Both cohorts were stratified by age (18-54 and ≥55 years) to align with age strata in ongoing COVID-19 vaccine efficacy trials. ## Methods ### Data analyses Symptoms recorded within three and seven days of symptom onset were included in the analyses (see **Supplementary Table 1** for complete list of symptoms and corresponding questions participants were asked). Analysis of symptoms within the first three days is key to enable testing for SARS-CoV-2 soon after symptom onset while viral load is highest. An additional buffer for inclusion of symptoms within seven days was also used, which may be important to detect development of lower respiratory tract signs indicative of pneumonia. Anosmia and ageusia were considered one symptom in the reporting app. Participants were classified as symptom-screening positive when they recorded at least one of the symptoms in the symptom combination concerned. This was compared with self-reported RT-PCR results considered the gold standard for COVID-19 case detection. If multiple positive RT-PCR test results were recorded for an individual, only the first was included. A COVID-19 case was defined as a newly symptomatic individual with a first ever positive RT-PCR test result. For individual symptoms or symptom combinations, three evaluation parameters were considered, taking disease status to be a positive RT-PCR test: 1) sensitivity, computed as the percentage of COVID-19 positive individuals correctly identified, 2) specificity, calculated as the percentage of individuals correctly classified as COVID-19 negative, and 3) the reciprocal of precision, that is the number of RT-PCR tests needed to identify one RT-PCR positive COVID-19 case (i.e. Tests Per Case [TPC]). ### Multi-objective evolutionary optimization As sensitivity and specificity of a given symptom combination represent conflicting objectives, a multi-objective evolutionary algorithm (MOEA) was used to generate optimal symptom combinations from the data, each characterised by a good trade-off between specificity and sensitivity. Optimisation problems with multiple objectives have a set of optimal solutions (i.e., Pareto-optimal solutions) rather than one single optimal solution. No Pareto-optimal solution is better than the other without further information on the specific objective to be addressed. For MOEA, we employed the well-known NSGAII 17 developed in the python package pymoo v0.4.2.1. The optimal set of parameters were derived through experimenting with different values (see **Supplementary Table 2** for parameter information). The training and validation datasets were used to generate and evaluate the Pareto-optimal symptom combinations (referred to as data-inferred symptom combinations). ### Evaluation of individual symptoms and symptom combinations Sensitivity, specificity, and TPC were evaluated for each individual symptom and symptom combinations using the validation dataset. We considered symptom combinations derived from both clinical experience/guidance (i.e., clinically inferred symptom combinations) and generated from the data using the MOEA (i.e., data-inferred symptom combinations). All evaluations were repeated on the US-replication cohort and on the data stratified by age. For clinically-inferred symptom combinations we evaluated: 1) respiratory symptoms (cough, dyspnoea), 2) WHO-defined pneumonia symptoms (cough, dyspnoea, fever), 3) COVID-19 specific symptoms as defined by Public Health England (PHE) (fever, cough, dyspnoea, anosmia/ageusia), and 4) extended symptoms (fever, cough, dyspnoea, anosmia/ageusia, fatigue, headache). This latter category was added post-hoc after exploration of the app data indicated high sensitivity of headache and fatigue in other contexts.18 Regarding data-inferred symptom combinations, among all the generated combinations, we evaluated the one with highest sensitivity, the one with a sensitivity of ∼90%, and the one characterised by a specificity of ∼50%, which is of interest from a clinical standpoint. ## Results A total of 122,305 individuals were included in the UK-discovery cohort, of which 1,202 tested COVID-19 positive. In the US-replication cohort, 3,162 individuals were included, of which 79 tested COVID-19 positive. The patient selection flow charts are displayed in **Supplementary Figure 1** and **2. Table 1** shows the demographic characteristics of the population. View this table: [Table 1.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T1) Table 1. Demographics of study population ### Evaluation of individual symptoms The sensitivity, specificity, and TPC for each individual symptom reported within three and seven days of symptom onset are displayed in **Table 2**. Using the UK-discovery cohort, the individual symptoms with the highest sensitivity in both three- and seven-day analyses were headache and fatigue (67% and 65% for three-day analysis and 75% and 78% for seven-day analyses). Similar results were obtained with data from the US-replication cohort and when data were stratified by age. The sensitivity of anosmia/ageusia in the UK-discovery cohort was only 22% and 49% in the three- and seven days analyses, respectively. Anosmia/ageusia, however, had the lowest TPC (20 and 10 for three- and seven-day analyses, respectively). View this table: [Table 2.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T2) Table 2. Sensitivity, specificity, and TPC for each individual symptom computed on the UK-discovery cohort These results are confirmed by **Figure 1**, which displays the frequency of the symptoms for the UK-discovery cohort for both COVID-19 positive and negative cases. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/02/08/2020.11.23.20237313/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/F1) Figure 1. Symptom frequency for COVID-19 negative (left) and COVID-19 positive (right) cases ### Evaluation of symptom combinations The sensitivity, specificity, and TPC of both clinically- and data-inferred symptom combinations, computed on the UK-validation and US-replication cohorts, and reported within three and seven days of symptom onset are displayed in **Table 3**. View this table: [Table 3.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T3) Table 3. Sensitivity, specificity, and TPC for the clinically and data-inferred combinations of symptoms, computed on the held-out validation dataset Cough or dyspnoea were reported by 46% of individuals positive for COVID-19 within the first three days of symptom onset. The addition of fever (i.e., WHO-defined pneumonia symptom combination) increased sensitivity to 60%, while the further addition of anosmia/ageusia (i.e., PHE COVID-19 specific symptom combination) increased sensitivity to 69%. When headache and fatigue are added, (i.e., extended symptom combination) the proportion of COVID-19 cases identified increased to 92% but the TPC doubled compared to the respiratory symptom combination (42 versus 85). Similarly, within seven days of symptom onset, COVID-19 specific and extended symptom combination were reported in 84% and 97% of RT-PCR positive cases, at the cost of 42 and 81 TPC, respectively. Similar results were obtained when data were stratified by age. The sensitivity estimates from the US-replication cohort were higher for all four combinations; extended symptom combination estimates reached 96% and 99% for the three- and seven-day analyses, respectively. On the contrary, the specificity decreased to 21% and 18%, although TPC values were lower for the US-replication cohort. Among data-inferred symptom combinations, the one with highest sensitivity (fatigue, anosmia/ageusia, cough, diarrhoea, headache, and sore throat) identified 96% and 99% of RT-PCR positive COVID-19 cases and required 96 and 92 TPC in the three- and seven-day analyses, respectively. The sensitivity results were similar for the US-replication cohort and by age. However, the number of tests needed for those aged ≥55 years increased by 30% for both the three-day and seven-day analyses. **Figure 2** displays the three data-inferred symptom combinations for both three- and seven-day analyses. Anosmia/ageusia were included in all three symptom combinations at both time points, fatigue was included in all symptom combinations for the three-day analyses, and cough for the seven-day analyses. Headache was slightly more important when symptoms were recorded within three days of onset. Diarrhoea as an individual symptom was not predictive of a positive COVID-19 RT-PCR result but became predictive when associated with other symptoms. All the Pareto-optimal symptom combinations generated by the MOEA are displayed in **Figure 3**. Each point (solution) of the Pareto corresponds to a certain symptom combination with a related sensitivity, specificity, and TPC (see **Supplementary Table 4** and **5** for the complete list of solutions for three- and seven-day analyses, respectively). These generated symptom combinations achieved similar values of sensitivity and specificity for the UK-training, UK-validation, and US-replication cohorts, thus confirming the validity of this methodology. Moreover, results were also confirmed for the two age groups. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/02/08/2020.11.23.20237313/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/F2) Figure 2. Combination of symptoms with highest sensitivity, sensitivity ∼ 90%, and specificity ∼50% ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/02/08/2020.11.23.20237313/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/F3) Figure 3. Pareto of optimal subset generated by the multi-objective evolutionary algorithm for three- and seven-day analyses **Figure 4** displays the frequency of symptoms selected in symptom combinations with a sensitivity ≥90%. Fatigue, cough, and anosmia/ageusia were present in most symptom combinations with high specificity. Diarrhoea was selected ∼60% of the time for the three-day analyses. ![Figure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/02/08/2020.11.23.20237313/F4.medium.gif) [Figure 4.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/F4) Figure 4. Percentage of a symptom’s appearance in symptom combinations with sensitivity ≥90 % Each point represents a subset of symptoms characterised by a different trade-off between sensitivity and specificity. ## Discussion We present data from, what is to our knowledge, the largest community-based COVID-19 symptom cohort study with the aim to quantify the contribution of various symptoms and symptom combinations associated with COVID-19 to RT-PCR positive case-finding. COVID-19 symptoms and RT-PCR test results were collected prospectively which allowed us to select newly symptomatic individuals and simulate a clinical trial situation in which RT-PCR tests are typically conducted within three days after symptom onset. We confirm the significance of symptoms (fever, cough, anosmia/ageusia) widely considered important for triggering a RT-PCR test and extend this to include additional symptoms (fatigue, sore throat, headache, diarrhoea). The proposed approach enables the selection of symptom combinations to maximise the capture of cases without overwhelming laboratory capacity. Our findings may help to optimise the choice of triggering symptoms for diagnostic work-up in COVID-19 vaccine efficacy trials or in a wider public health setting. In an efficacy trial, it is important to capture all COVID-19 cases with pulmonary involvement as signs/symptoms of pneumonia define moderate or severe COVID-19. Therefore, the signs/symptoms that characterise WHO-defined COVID-19 pneumonia (fever, cough, dyspnoea, tachypnoea) should trigger diagnostic work-up in a trial participant.19 Additionally, anosmia/ageusia have the highest sensitivity of all reported COVID-19 symptoms.9,20 Although our findings support the inclusion of these COVID-19 specific symptoms, they also show that this combination correctly identified only 69% and 83% of COVID-19 cases in the three- and seven-days analyses. This has important implications in terms of cases missed as the COVID-specific symptoms align with the current PHE definition of a possible COVID-19 case.21 We found that the addition of headache and fatigue (i.e., extended symptoms) increased the proportion of COVID-19 cases correctly identified to 92% but also almost doubled the TPC (from 47 to 85). Thus, an increase in sensitivity comes at a cost. Application of MOEA identified fatigue, anosmia/ageusia, cough, diarrhoea, headache, and sore throat as the symptom combination with the highest sensitivity in three- and seven-day analyses. Diarrhoea and sore throat were identified as symptoms that may increase case finding in an efficient way, in addition to those symptoms already considered important for triggering an RT-PCR test. In situations where there is a limited testing capacity, we provide a range of optimal symptom combinations that could be used, given different target numbers of TPC identified. This finding may prove useful for COVID-19 vaccine developers or in public health settings when deciding which symptoms should trigger testing to optimise financial and logistical resource utilisation. Importantly, all the symptoms that constitute the combination with the highest sensitivity have been included as triggering symptoms in publicly available clinical trial protocols of ongoing vaccine efficacy trials.9-14 Few studies have been published that assess COVID-19 symptoms in community-based cohorts. Menni et al. presented results using data generated from this COVID-19 Symptom Study app; however, the aim was different and only data from March-April 2020 were included.22 We extend these data to September 2020 and, importantly, consider the results from the perspective of a potential COVID-19 vaccine developer. Menni et al. suggest anosmia/ageusia, fatigue, persistent cough, and loss of appetite might together identify individuals with COVID-19.22 A separate COVID-19 symptom app from Germany suggests nausea and vomiting have a stronger predictive value for COVID-19 infection than symptoms such as sore throat or persistent cough.23 Thus, both studies identify gastrointestinal symptoms as important in identifying cases of COVID-19. Our study reports similar findings with diarrhoea found to be important to case finding. More recently, in another community-based observational study, sensitivity, specificity, and positive and negative predictive values were reported for retrospectively collected symptoms and symptom combinations that occurred during the 14-day period prior to screening for SARS-CoV-2 infection in a US seroprevalence study.24 The two symptom clusters most associated with SARS-CoV-2 infection were: 1) ageusia, anosmia, and fever, and 2) shortness of breath, cough, and chest pain. In our study, dyspnoea was rarely and chest pain never selected as part of an efficient symptom combination likely due to dyspnoea often occurring later in the disease course.25 The sensitivity of dyspnoea increased in the seven-day compared to three-day analyses. However, the importance of dyspnoea as a symptom of pulmonary involvement makes it a critical triggering symptom in vaccine efficacy trials. Tachypnoea, which is included in the WHO-defined definition for pneumonia, was not captured as a symptom in the app per se; however, it likely co-occurs with dyspnoea. Headache and diarrhoea were more likely to be selected in the three-day scenario and fever during the seven-day scenario again, reflecting different timings of symptoms in the disease course. The sensitivity of symptoms and various clinically inferred symptom combinations were similar for the age groups (18-54 and ≥55 years); however, the TPC was higher in the ≥55 years age group. This suggests self-reporting may work better for younger than older individuals. The sensitivity, specificity, and TPC computed on the US-replication cohort were higher than for the UK-discovery cohort possibly due to different testing practices and public health measures adopted in each country. It will be important for these findings to be validated in low- and middle-income country (LMIC) settings as COVID-19 vaccine efficacy trials are likely to be conducted in high income countries as well as LMICs. Vaccine developers should take into account regional considerations such as background incidence of co-infection and other trial-related aspects when interpreting these results. This study has many strengths, including the large sample size and cost-effectiveness of the data source. Also, our study is community-based and adds important data as most studies that have assessed symptoms in COVID-19 have involved hospital-based populations. Some limitations, however, also need consideration. First, the results are based on data self-reported through a mobile app and therefore biased towards people with smartphone access. However, the app included a feature to enable reporting on behalf of someone else given their consent. Second, reported test results were not externally verified, however, antigen tests were not available during the study period, thus minimising risk of participant confusion regarding precise swab tests. As the precise RT-PCR used was not recorded and likely varied between participants, false positive rates were not known and results taken at face value. A further limitation is that app users may not be representative of the wider population. Finally, these data were generated in the spring and summer months when the incidence of concurrent respiratory infections (e.g., influenza) is low. The latter may have implications for trials conducted in winter. In summary, we confirm the significance of symptoms widely recommended for triggering RT-PCR and identified additional symptom combinations to enable efficient trade-off between the number of positive cases detected and tests needed. Our findings may help optimise the choice of triggering symptoms for diagnostic work-up in COVID-19 vaccine efficacy trials and also have wider public health implications. ## Data Availability Data collected in the COVID-19 Symptom Study smartphone application are being shared with other health researchers through the UK National Health Service-funded Health Data Research UK (HDRUK) and Secure Anonymised Information Linkage consortium housed in the UK Secure Research Platform (Swansea, UK). Anonymised data are available to be shared with HDRUK researchers according to their protocols in the public interest (https://web.www.healthdatagateway.org/dataset/fddcb382-3051-4394-8436-b92295f14259). US investigators are encouraged to coordinate data requests through the Coronavirus Pandemic Epidemiology Consortium (https://www.monganinstitute.org/cope-consortium). [https://web.www.healthdatagateway.org/dataset/fddcb382-3051-4394-8436-b92295f14259](https://web.www.healthdatagateway.org/dataset/fddcb382-3051-4394-8436-b92295f14259) ## Conflicts of interests ### Potential conflicts of interest JW, RD, JCP, and AM are employees of Zoe Global Ltd. ATC reports grants from Massachusetts Consortium on Pathogen Readiness during the conduct of the study, personal fees from Pfizer Inc., and grants and personal fees from Bayer Pharma; CEPI (authors AC, JG, JPC, AEL) funds clinical trials of COVID-19 vaccines. All other authors declare no competing interests. ## Funding This work was supported by Zoe Global Limited; Department of Health; Wellcome Trust; Engineering and Physical Sciences Research Council (EPSRC); National Institute for Health Research (NIHR); Medical Research Council (MRC); Alzheimer’s Society; Massachusetts Consortium for Pathogen Readiness (MassCPR); and Coalition for Epidemic Preparedness Innovations (CEPI). ## Supplementary tables View this table: [Supplementary Table 1.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T4) Supplementary Table 1. List of self-reported symptoms and corresponding question used in the reporting app View this table: [Supplementary Table 2.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T5) Supplementary Table 2. NSGAII parameters View this table: [Supplementary Table 3.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T6) Supplementary Table 3. Sensitivity, specificity, and TPC for the four clinically inferred symptom combinations computed on the UK-discovery cohort View this table: [Supplementary Table 4.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T7) Supplementary Table 4. Pareto of optimal combination of symptoms for three-day analysis computed on the UK-discovery training data, ordered by decreasing sensitivity View this table: [Supplementary Table 5.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/T8) Supplementary Table 5. Pareto of optimal combination of symptoms for seven-day analysis computed on the UK-discovery training data, ordered by decreasing sensitivity ## Supplementary Figures ![Supplementary Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/02/08/2020.11.23.20237313/F5.medium.gif) [Supplementary Figure 1.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/F5) Supplementary Figure 1. Flow diagram of user selection for the UK-discovery cohort ![Supplementary Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/02/08/2020.11.23.20237313/F6.medium.gif) [Supplementary Figure 2.](http://medrxiv.org/content/early/2021/02/08/2020.11.23.20237313/F6) Supplementary Figure 2. Flow diagram of user selection for the US-replication cohort ## Acknowledgements Zoe provided in kind support for all aspects of building, running and supporting the app and service to all users worldwide. CEPI provided funding for the analysis of the data. Support for this study was provided by the NIHR-funded Biomedical Research Centre based at GSTT NHS Foundation Trust. Investigators also received support from the Wellcome Trust, the MRC/BHF, Alzheimer’s Society, EU, NIHR, CDRF, and the NIHR-funded BioResource, Clinical Research Facility and BRC based at GSTT NHS Foundation Trust in partnership with KCL, the UK Research and Innovation London Medical Imaging & Artificial Intelligence Centre for Value Based Healthcare, the Wellcome Flagship Programme (WT213038/Z/18/Z), the Chronic Disease Research Foundation, and DHSC. DAD is supported by the National Institute of Diabetes and Digestive and Kidney Diseases K01DK120742 and by the American Gastroenterological Association AGA-Takeda COVID-19 Rapid Response Research Award (AGA2021-5102). ATC was supported in this work through a Stuart and Suzanne Steele MGH Research Scholar Award. The Massachusetts Consortium on Pathogen Readiness (MassCPR) and Mark and Lisa Schwartz supported MGH investigators (LHN, DAD, ADJ, ATC). * Received November 23, 2020. * Revision received February 8, 2021. * Accepted February 8, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NoDerivs 4.0 International), CC BY-ND 4.0, as described at [http://creativecommons.org/licenses/by-nd/4.0/](http://creativecommons.org/licenses/by-nd/4.0/) ## References 1. 1.Hodgson SH, Mansatta K, Mallett G, Harris V, Emary KRW, Pollard AJ. What defines an efficacious COVID-19 vaccine? A review of the challenges assessing the clinical efficacy of vaccines against SARS-CoV-2. Lancet Infect Dis 2020; S1473-3099(20):30773–8. 2. 2.Corey L, Mascola JR, Fauci AS, Collins FS. A strategic approach to COVID-19 vaccine R&D. Science 2020; 368(6494):948–950. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNjgvNjQ5NC85NDgiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMi8wOC8yMDIwLjExLjIzLjIwMjM3MzEzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 3. 3.Polack FP, Thomas SJ, Kitchin N, et al. Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. New England Journal of Medicine, December 2020, doi = {10.1056/NEJMoa2034577}. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2034577&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33301246&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F08%2F2020.11.23.20237313.atom) 4. 4.Moderna Announces Primary Efficacy Analysis in Phase 3 COVE Study for Its COVID-19 Vaccine: [https://investors.modernatx.com/node/10421/pdf](https://investors.modernatx.com/node/10421/pdf) [Accessed 9th December 2020]. 5. 5.AZD1222 vaccine met primary efficacy endpoint in preventing COVID-19: [https://www.astrazeneca.com/media-centre/press-releases/2020/azd1222hlr.html](https://www.astrazeneca.com/media-centre/press-releases/2020/azd1222hlr.html) [Accessed 9th December 2020]. 6. 6.Petersen I, Phillips A. Three Quarters of People with SARS-CoV-2 Infection are Asymptomatic: Analysis of English Household Survey Data. Clin Epidemiol. 2020;12:1039–1043. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2147/CLEP.S276825&link_type=DOI) 7. 7.Pormohammad A, Ghorbani S, Khatami A, et al. Comparison of influenza type A and B with COVID-19: A global systematic review and meta-analysis on clinical, laboratory and radiographic findings. Rev Med Virol. 2020:e2179. 8. 8.Wiersinga JW, Rhodes A, Cheng AC, et al. Pathophysiology, transmission, diagnosis, and treatment of coronavirus disease 2019 (COVID-19): a review. JAMA. 2020;324(8):782–793. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.12839.e4&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F08%2F2020.11.23.20237313.atom) 9. 9.Haehner A, Draf J, Drager S, de With K, Hummel T. Predictive value of sudden olfactory loss in the diagnosis of COVID-19. ORL J Otor. Relat Spec 2020;82(4):175–180. 10. 10.A Phase 3, Randomised, Stratified, Observer-Blind, Placebo-Controlled Study to Evaluate the Efficacy, Safety, and Immunogenicity of mRNA-1273 SARS-CoV-2 Vaccine in Adults Aged 18 Years and Older. [https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf](https://www.modernatx.com/sites/default/files/mRNA-1273-P301-Protocol.pdf) 11. 11.A phase 1/2/3, placebo-controlled, randomised, observer-blind, dose-finding study to evaluate the safety, tolerability, immunogenicity, and efficacy of SARS-CoV-2 RNA vaccine candidates against COVID-19 in healthy. individuals [https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001\_Clinical\_Protocol\_0.pdf](https://pfe-pfizercom-d8-prod.s3.amazonaws.com/2020-09/C4591001_Clinical_Protocol_0.pdf) 12. 12.A Randomised, Double-blind, Placebo-controlled Phase 3 Study to Assess the Efficacy and Safety of Ad26.COV2.S for the Prevention of SARS-CoV-2-mediated COVID-19 in Adults Aged 18 Years and Older. [https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol](https://www.jnj.com/coronavirus/covid-19-phase-3-study-clinical-protocol) 13. 13.A Phase III Randomized, Double-blind, Placebo-controlled Multicenter Study in Adults to Determine the Safety, Efficacy, and Immunogenicity of AZD1222, a Non-replicating ChAdOx1 Vector Vaccine, for the Prevention of COVID-19. [https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001\_CSP-v2.pdf](https://s3.amazonaws.com/ctr-med-7111/D8110C00001/52bec400-80f6-4c1b-8791-0483923d0867/c8070a4e-6a9d-46f9-8c32-cece903592b9/D8110C00001_CSP-v2.pdf) 14. 14.A Phase 3, Randomised, Observer-blinded, Placebo-Controlled Trial to evaluate the Efficacy and Safety of a SARS-CoV-2 Recombinant Spike Protein Nanoparticle Vaccine (SARS-CoV-RS) with Matrix-M1 Adjuvant in Adult participants 18-84 years of Age in the United Kingdom. [https://www.novavax.com/download/files/protocols/2019nCoV302Phase3UKVersion2FinalCleanRedacted.pdf](https://www.novavax.com/download/files/protocols/2019nCoV302Phase3UKVersion2FinalCleanRedacted.pdf) 15. 15.Voysey M, Costa SU, Madhi SA, et al. Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. Lancet 2020; DOI:[https://doi.org/10.1016/S0140-6736(20)32661-1](https://doi.org/10.1016/S0140-6736(20)32661-1) 16. 16.Drew DA, Nguyen LH, Steves CJ, et al. Rapid implementation of mobile technology for real-time epidemiology of COVID-19. Science 2020;368(6497):1362–1367. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEzOiIzNjgvNjQ5Ny8xMzYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDIvMDgvMjAyMC4xMS4yMy4yMDIzNzMxMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 17. 17.Deb K, Pratap A, Agarwal S, Meyarivan T. A fast and elitist multi-objective genetic algorithm: NSGA-II IEEE. Transactions on Evolutionary Computation 2002;6:182–97. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1109/4235.996017&link_type=DOI) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000175082800006&link_type=ISI) 18. 18.Sudre CH, Lee K, Lochlainn MN. Symptom clusters in Covid19: A potential clinical prediction tool from the COVID Symptom study App. medRxiv Jun 16; 2020.06.12.20129056. doi: 10.1101/2020.06.12.20129056. Preprint [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4wNi4xMi4yMDEyOTA1NnYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDIvMDgvMjAyMC4xMS4yMy4yMDIzNzMxMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 19. 19.Clinical Management of COVID-19 (WHO interim guidance): [https://www.who.int/publications/i/item/clinical-management-of-covid-19](https://www.who.int/publications/i/item/clinical-management-of-covid-19) (Accessed November 22, 2020). 20. 20.Agyeman AA, Chin KL, Landersdorfer CB. Smell and Taste Dysfunction in Patients With COVID-19: A systematic review and meta-analysis. Mayo Clin Proc. 2020;95(8):1621–1631. 21. 21.Public Health England. [https://www.gov.uk/government/publications/wuhan-novel-coronavirus-initial-investigation-of-possible-cases/investigation-and-initial-clinical-management-of-possible-cases-of-wuhan-novel-coronavirus-wn-cov-infection#criteria](https://www.gov.uk/government/publications/wuhan-novel-coronavirus-initial-investigation-of-possible-cases/investigation-and-initial-clinical-management-of-possible-cases-of-wuhan-novel-coronavirus-wn-cov-infection#criteria) (Accessed December 11th 2020). 22. 22.Menni C, Valdes AM, Freidin MB, et al. Real-time tracking of self-reported symptoms to predict potential COVID-19. Nat Med. 2020;26(7):1037–1040. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41591-020-0916-2&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F08%2F2020.11.23.20237313.atom) 23. 23.Zens M, Brammertz A, Herpich J, et al. App-based tracking of self-reported COVID-19 symptoms: analysis of questionnaire data. J Med Internet Res. 2020;22(9):e21956. 24. 24.Dixon BE, Wools-Kaloustian K, Fadel WF, et al. Symptoms and symptom clusters associated with SARS-CoV-2 infection in community-based populations: Results from a statewide epidemiological study. medRxiv Oct 22;2020.10.11.20210922. doi: 10.1101/2020.10.11.20210922. Preprint [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyMC4xMC4xMS4yMDIxMDkyMnYyIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjEvMDIvMDgvMjAyMC4xMS4yMy4yMDIzNzMxMy5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 25. 25.Tang D, Comish P, Kang R. The hallmarks of COVID-19 disease. PLoS Pathog 2020;16(5):e10085