Self-rated smell ability enables highly specific predictors of COVID-19 status: a case control study in Israel

Aim and setting

This prospective study compared symptoms in real time (RT) polymerase chain reaction (PCR)-tested COVID-19 positive and COVID-19 negative patients. Patients having PCR-test results (positive or negative) were recruited via social media (Twitter and Facebook) and word of mouth from March 2020 to May 2020 and interviewed from April 2020 to June 2020. The cohort comprised 224 Israeli patients aged ≥18 years (Figure 1) Israeli patients. The participants were not aware that the questionnaire will include smell and taste loss symptoms prior to their agreement to partake in the study. Informed consent was obtained from all participants. The study was approved by the Hadassah Medical Center Helsinki Committee (permit number 0236-20-HMO).

• Download figure
• Open in new tab

Figure 1. Flow chart of patients’ selection for case control study.

General design

The interviews were carried out over the phone. The questionnaire is based on questions compiled by physicians and scientists in the Global Consortium for Chemosensory Research, GCCR[7]. The full questionnaire is included in Supplementary data and has five parts: 1) General information (e.g., age, gender); 2) Medical history (e.g., medical conditions, medications, changes in taste/smell in the past, pregnancy, contact with a confirmed patient); 3) Current illness: 23 physical signs and symptoms, including binary question (yes/no) on smell, taste and chemesthesis (cooling, burning, tingling sensation), PCR swab test results and dates, date of exposure to confirmed COVID-19 patient, subjective recovery feeling; 4) Smell: Participants were instructed to rate their sense of smell/ taste and the degree of their nose blockage on a scale from 1-10 (1 corresponding to “no sense of smell” and 10 to excellent sense of smell) and similarly rate the ability to breathe through the nose before/during/after illness. Blocked nose rating was used to test the plausible hypothesis that it causes a change of smell; 5) Taste (e.g., rating of taste ability before/during/after illness, as described for smell), experience of strange/bad taste in the mouth, change in sensitivity to irritants (chemesthesis) and change in basic taste modalities – sweet, salty, sour, bitter, each elicited by non-volatile compounds via specific receptors or channels expressed in dedicated taste receptor cells [19]. The fifth basic taste modality, umami or savory, was not used because it does not have a Hebrew translation. “Other” taste was available as an additional optional answer. Data was kept in Compusense Cloud on-line software (Compusense Inc., Guelph, ON, Canada).

Statistical analysis

Log-odds for the individual symptoms were calculated over the full dataset. Confidence intervals and p-values for the log-odds were estimated from the glm function using the logistic link implemented in the statistical software R (https://www.r-project.org/).

Classifiers were trained from the reported symptoms to evaluate the separation between COVID-19 positive and COVID-19 negative patients. The classifiers were trained on a random subset of 2/3 of the data (the training set, 148 samples), and evaluated on the remaining samples (the test set). Sampling of the train and test sets was stratified by COVID-19 status. We trained the classifier on the full symptom matrix: All symptoms of question 23 in the questionnaire (see Supplementary) were included, except “no symptoms” or “other”. All eye symptoms were combined to “Eye symptoms”. Also added were quantitative questions for taste, smell, and nose blockage (rating before the illness minus rating during the illness, questions 31, 35, 37, 38, 40, 41 in the questionnaire) and chemesthesis (question 45) “Coated tongue”, “Dizziness”, “Ears pressure”, “Eye burn”, “Eye discharge”, “Hearing change”, “Lacrimation”, and “Vision changes” were removed since these symptoms were reported by less than 10% of the subjects. The classifiers were trained as penalized logistic regressions, using the elastic net algorithm (α = 0.5 implemented in the glmnet package in the R environment. This regression method encourages sparse coefficient vectors, meaning that it is suitable in situations where only few coefficients are non-zero. The regularization parameter (lambda) was initially set using cross-validation, but then increased until the model included no more than six symptoms. For classifiers based on a single symptom, no regularization was used.

Classifiers were evaluated using the hold-out test set. The score from the classifier was thresholded at zero, so that patients with score exceeding 0 were called positive by the classifier. Sensitivity (predicting COVID-19 positives correctly) and specificity (predicting COVID-19 negatives correctly) metrics were calculated from the following formulas: in which TP (TN) are the COVID-19 positives (negatives) classified correctly, and FN (FP) are the COVID-19 positives (negatives) classified incorrectly. Due to our balanced sample, accuracy is the average of sensitivity and specificity. We further computed the accuracy metrics that account for the expected proportion of positive cases in the tested population, namely the positive predictive value (PPV) and the negative predictive value (NPV). The scores obtained from the logistic classifier (s) were translated into probability to be positive (P) by adjusting for the proportion of COVID+ out of the tests (π) according to the following formulas:

We take π to be 0.08 because that was the proportion during data collection.

The ROC curve corresponds to true-positive and false-positive rates for different values of the threshold; the curve and the area under the ROC curve (AUC), which measures the degree of separability between positive and negative scores, were estimated using the pROC package[20].

Sample size calculation

The sample size was calculated to allow detecting differences in smell-loss or taste-loss prevalence between COVID-19 positive and negative populations. Based on previous research [e.g. [21]], we used conservative estimates of 60% prevalence in the positive population and 35% prevalence in the negative population. Power was estimated by Monte Carlo simulations, namely repeatedly (b=1000) resampling from two Binomial distributions corresponding to the positive population and the negative population. Assuming 100 individuals are assigned to each group, and a two-sided t-test is used, the probability of detection (power) is 92%. To be on the conservative side, we used somewhat larger samples.

Patients’ characteristics

Completed questionnaires were obtained from 112 COVID-19 positives and 112 COVID-19 negatives. The median age of the respondents was 35 ± 12 years for positives and 37 ± 12 years for negatives (mean ± SD) years. The positives group included more men (64%), while the negatives group was more balanced (48% males). Seven patients classified as hospitalized (received respiratory support during their hospitalization and / or were hospitalized in the intensive care unit) and the rest 217 were classified as ambulatory patients.

Patients’ symptoms

Signs and symptoms that appeared in the binary part of the questionnaire (Supplementary Material, question 23 of the full questionnaire) and were found to occur in at least 10% of the positive patients are summarized in Table 1. A few symptoms, including dry cough and sore throat, were prevalent in COVID-19 positives, but even more so in the negatives control sample.

Smell change, taste change, change in chemesthetic ability (perceiving spicy, tingling or cooling sensations) and muscle ache were significantly more prevalent in COVID-19 positive as compared to COVID-19 negative patients (68%, 72%, 31%, 62% vs. 8.0%, 18%, 6%, 34%, respectively) (Table 1). Other CDC recognized symptoms[22], such as lack of appetite, fever, and diarrhea were approximately twice or three times more common among positives than negatives.

Nausea or vomiting, although considered a COVID-19 symptom, were not more common among COVID-19 positive as compared to COVID-19 negative patients. By contrast, lack of appetite, despite not being included as an “official” CDC symptom[22], was found to be significantly more common in COVID-19 positive patients.

Taste and smell change often, but not always, together: Figure 2A shows the distribution of reports on taste and smell changes. Change in both smell and taste perception was reported by 63% of positive patients and only 6% of negative patients. 4% of positive patients experienced only smell change and 9% reported taste change with no smell change.

• Download figure
• Open in new tab

Figure 2. Smell, taste, and taste modalities changes during disease.

A. The pie chart presents prevalence of smell and taste changes in positive and negative patients, occurring together or separately. Numbers indicate percentage of COVID-19 positives and negatives reporting taste or smell loss (blue and green respectively), both taste and smell loss (turquoise) or neither (seashell). B. The prevalence of the four taste modalities in COVID-19 positive and negative patients. COVID-19 positives are represented in purple and COVID-19 negatives are represented in orange

Approximately 60% of the positive patients reported impairment of at least one of the four taste modalities (sweet, salty, sour, and bitter) compared to only ∼6% among the negative patients (Figure 2B). In COVID-19 positive patients with taste impairment, all four taste modalities were usually impaired. In 31% of the positive patients and in 90% of the negative ones, no taste modality was impaired.

Chemosensory changes rating and details

In addition to the binary questions, the participants were asked to rate their smell and taste senses before and during their illness on a 1-10 scale.

As is seen in Figure 3, the change in smell and taste ability during disease, compared to a self-reported individual baseline before the disease, differs greatly between positive and negative patients. Both taste and smell changes were significantly greater for COVID-19 positive patients compared to COVID-19 negative patients (4.5±3.6 and 4.9±3.8 vs. 0.5±1.5 and 0.4±1.5, mean ± SD, p<0.0001 and p<0.0001, respectively). When considering only patients who reported taste or smell changes (answered “yes” to the respective binary questions), the averages in positive patients were 6.3±2.6 for taste and 7.1±2.4 for smell, compared to negative patients with 2.1±2.3 for taste and 4.8±2.6 for smell (among patients with any change in taste p<0.0001, in smell p<0.0001).

• Download figure
• Open in new tab

Figure 3. Degree of taste smell and nose blockage in COVID-19 positives and negatives.

A. Histogram showing the change in rating during the illness minus rating before the illness for taste, smell, and nose blockage. No change is coded 0 and the highest change is 9. COVID-19 positives are represented in purple and COVID-19 negatives are represented in orange. B. Table of mean ± SD for COVID-19 positives and negatives in general and for those reporting changes of taste or smell. Scores for taste, smell, and nose blockage were evaluated on a 1-10 scale. P values for the difference in the magnitude of change between COVID-19 positives and negatives was calculated using a two-sided t-test.

We next set out to check which combination of symptoms will be most useful for differentiating between COVID-19 positive and COVID-19 negative diagnosis. To that end, several classifiers were trained based on 66% of the sample and evaluated on 34% that was kept as a holdout set. The process of selection of descriptors is outlined in Figure 4. Relevant symptoms (n=30) were included as possible descriptors for the classifiers, and the elastic-net penalization was increased until no more than six symptoms were included in the model (number limited for practicality). The effect of excluding or including a particular symptom was evaluated in order to understand the importance of separating taste from smell and using binary vs. quantitative measures for each (Figure 4).

• Download figure
• Open in new tab

Figure 4. Descriptors selection process for classifiers.

The flow chart depicts total symptoms selected for the classifiers. Upon limiting the number of descriptors for the classifiers and excluding chemosensory symptoms as described, Classifier 1 was created. Different combinations of symptoms established better classifiers than Classifier 1, those using quantitative questions exhibiting better performance than those using binary ones. The classifier using “smell” and “taste” as separate descriptors, rather than “smell or taste” as a single joint descriptor showed better performance. The “Basic” + smell only descriptor outperformed the “Basic” + taste only descriptor, resulting in Classifier 2. Finally, the smell only descriptor was tested alone without all other “Basic” symptoms, resulting in Classifier 3.

The results of the evaluation on the holdout set are summarized in Supplementary Table 1, and classifiers 1-3 can be seen in Figure 5. Classifiers that did not use chemosensory symptoms had poor performance (AUC 0.60, black curve, classifier 1, and additional classifiers (Table S1). Adding the quantitative smell-change symptom (maroon curve, Classifier 1) is sufficient to outperform all other classifiers (AUC 0.83). Remarkably, using quantitative smell-change as a sole symptom (magenta curve, Classifier 3) resulted in a classifier that was nearly equally effective as Classifier 2 (AUC 0.81).

• Download figure
• Open in new tab

Figure 5.

A. ROC curves for the classifiers considered. B. Statistical parameters for the ROC curves. For the quantitative descriptors, the coefficients correspond to a descriptor scaled to the 0-1 range. C. Descriptors, their coefficients and intercept (fitted parameter) used in Classifiers 1, 2 and 3.

Adding taste change to Classifier 2 did not improve its performance, as it resulted in AUC of 0.82 (Classifier 7, Table S1). Furthermore, taste change as a sole descriptor resulted in AUC of 0.75 (Classifier 15, Table S1) and as an added descriptor to other “Basic” symptoms, it resulted in AUC of 0.76 (Classifier 13, Table S1). Thus, while there is a high correlation (0.82) between quantitative changes in smell and quantitative changes in taste, in our sample the smell change descriptor outperforms the taste descriptor. Using the quantitative smell and taste descriptors resulted in higher AUC’s than binary (yes/no) descriptors of these changes. For example, a binary smell descriptor used as a sole descriptor resulted in AUC of 0.78 (Classifier 16, Table S1), as compared to AUC of 0.81 using quantitative smell descriptor.

Scores according to the classifiers can be readily transformed into the probability for COVID-19, under additional factor: the known rate of positive tests. The rate of positive tests out of total PCR tests at the time of participants recruitment was calculated to be 8% based on data from Israeli Ministry of Health. By incorporating this factor (as described in the Methods), we were able to reach PPV of 0.68 and NPV of 0.97 for both classifiers. A probability calculator based on Classifiers 2, 3 and 13 (from Supplementary) is available in GITHUB (https://github.com/KimAsseo/Hadassah_COVID-19) and may be used for practical purposes in clinical life.

Study limitations

The method of patient recruitment is one of the limitations of this study: social media-based recruitment may limit participants’ representation as it targets mostly younger patients, with internet access and social media accounts. Word of mouth recruitment was used as well and contributes as well to creating a sample that is not necessarily representative of the general population. Male and female patients were not fully matched across positives (64% males) and negatives (48% males), in accord with higher % of males (56%) among COVID-19 patients in Israel.

Importantly, symptoms-based classifiers cannot capture asymptomatic COVID-19 patients. Therefore, low probability established with our classifiers should not be considered as a predictor of negative COVID-19 status. In other words, our classifiers are not SNOUT (‘Sensitive test when Negative rules OUT the disease’) but can definitely be referred to as ‘Specific test when Positive rules IN the disease’ (SPIN).

While chemosensory loss is a dominant feature of symptomatic COVID-19 patients, about 30% of them do not report such loss. Our sample was not large enough to include many positive patients in this subgroup, and further studies are needed to capture distinctive characteristics of COVID-19 patients with intact smell and taste.

Additionally, our sample was composed of light to moderately ill patients, thus the classifiers reported are not necessarily applicable to patients with severe forms of COVID-19. It should also be kept in mind that our data is specific to Israeli patients and reflect to some degree the criteria for PCR tests eligibility during the recruitment (fever and dry cough were sufficient for PCR test but change of smell and/or taste alone was not).

Another possible limitation of this study is the self-reporting, rather than objective testing method for data collection used. Clearly, the infectious nature of the disease cause data collection to be mostly based on self-reports[29], but classifiers based on brief objective examination of taste and smell symptoms may provide additional insights.

Lastly, RT-PCR is a commonly used diagnostic test for COVID-19 and for that reason it was used in the present study as a distinguishing criterion. Nevertheless, even this test may report false diagnosis (26%-37%)[30, 31]. Considering the inaccuracy of PCR tests, high scores from our classifiers may help to reinforce true positive results and possibly capture PCR false negative patients.