Sensitivity and specificity of prediction models based on gustatory disorders in diagnosing COVID-19 patients: a case-control study

Introduction

The novel coronavirus SARS-CoV-2 (severe acute respiratory syndrome coronavirus 2) is the cause of an ongoing pandemic. The outbreak of the novel coronavirus was first reported in the city of Wuhan in Hubei Province, China but rapidly went on to affect almost all countries worldwide.¹ The disease is characterised by fever, dry cough, dyspnea, chest pain and, in many cases, pneumonia; however, many other non-respiratory signs and symptoms have been described, such as general weakness, myalgia and arthralgia, or headache.²

COVID-19 may also have neurological manifestations, some of which may be due to the fact that coronaviruses enter the central nervous system via the peripheral nerves or olfactory bulb and olfactory sensory neurons.³ The latter route of viral invasion may be associated with smell disorders, which appear to be highly common in COVID-19 patients. For example, Moein et al. described smell impairment assessed with a well-validated test in 98% of patients, among whom 58% were either anosmic or severely microsmic,⁴ the frequency of self-reported olfactory disorders may be however lower. In a relatively large cohort of a multicentre European study, the frequency of self-reported olfactory dysfunction was 88%.⁵ In another study, the self-reported frequency of anosmia was 67.8% compared with 16% of SARS-CoV-2- negative subjects.⁶

Other non-respiratory manifestations of COVID-19 include signs and symptoms arising from the gastrointestinal tract, such as nausea, vomiting, diarrhoea and anorexia. Such symptoms have been found in a significant percentage (from below 10% to almost 70%) of patients.⁷ Taste disorders have also been reported, and although some reports regard single patients only, dysgeusia and ageusia appear to constitute two common symptoms of COVID-19, occurring in 10% to almost 90% of patients.^5,8,9,10 However, similar to olfactory dysfunction, most studies regarding taste disorders in SARS-CoV-2 positive patients published to date have been based on self-reported symptoms, and only one study has quantified gustatory dysfunction in patients, though not in healthy control subjects.¹¹

Therefore, the present study aimed to conduct a more precise assessment of gustatory function in COVID-19 patients. Specifically, disorders in sweet, sour, and salty and bitter tastes in a group of young, otherwise asymptomatic or mildly symptomatic subjects were quantitatively assessed and compared with a properly matched SAR-CoV-2-negative control group.

In addition, the sensitivity of the relatively expensive RT-PCR test in COVID-19 diagnosis, which is the gold standard diagnostic method for SARS-CoV-2 according to the World Health Organization (WHO) may be may be from different reasons (e.g. shallow swab instead of deep nasopharyngeal one) insufficient,, particularly for single tests.^12,13 The sensitivity of other diagnostic algorithms using CT scan images may be much higher; however, this procedure is also expensive, and x-ray exposure may lead to adverse effects.¹⁴ Both the above-mentioned approaches are time-consuming and are not point-of-care procedures. As a result, their suitability for large-scale disease screening is limited. Therefore, this study aimed to establish a reliable, sensitive and specific test to diagnose COVID-19 by assessing taste disorders. The publication was prepared according to the Standards for Reporting of Diagnostic Accuracy Studies (STARD 2015).¹⁵

Material and Methods

Main study

The main study was conducted from April to May 2020 on 92 students from Warsaw, Poland (all of those living in a dormitory) who fulfilled the inclusion (confirmed SARS-CoV-2 infection, aged over 18) and exclusion criteria (lack of informed consent, smell or taste disturbances lasting more than three months on the day of the test, any neurological disease, any disease or factor that in the opinion of an investigator may influence the tests) and who provided signed informed consent. Due to the epidemiological situation, after detecting a SARS-CoV-2 coronavirus outbreak in one of the Universities in Warsaw, Poland, students living together in the dormitory were tested for SARS-CoV-2 using the RT-PCR test of nasopharyngeal swab, according to the WHO standards. Students with a positive test result were isolated from those with negative test results and were transferred to isolation rooms in a specially adapted hotel. Students who tested negative were placed in individual rooms in the school dormitory. Students from both groups were placed under the supervision of the Central Clinical Hospital of the Ministry of the Interior and Administration in Warsaw, which had been transformed into an infectious hospital.

Eighty-eight students completed the study. Of these students, 51 were in the study group (100% included in the study) and 37 were in the control group (90% included in the study; four study subjects did not return the questionnaire). The study flow is shown in Figure 1. On the day of the examination, an additional swab was collected, and a PCR test performed in each student to confirm SARS-CoV-2 infection. Therefore, at the time of the data collection, the researchers knew the results of the first PCR test but not the second. Only one person from the control group tested positive for SARS-CoV-2 after re-examination. This result was confirmed with a PCR test, and the participant was transferred from the control group to the test group (therefore, the final number of participants in the control group was 36 and the study group was 52) for further statistical analysis. RT-PCR tests were conducted in the National Institute of Public Health - National Institute of Hygiene (NIPH-NIH), which is the main governmental public health research institute and the reference centre for the national network of sanitary epidemiological service.¹⁶ The GeneFinder COVID-19 PLUS RealAmp Kit was used, detecting RdRp, E and N genes, with a sensitivity of 10 copies of these genes, and cut-off point of ≤43 cycles for a positive result. The assessors of the PCR (reference) tests were not aware of any clinical information regarding participants or the flavour test results.

• Download figure
• Open in new tab

Figure 1.

Study flow diagram. Test 1 – sensitivity test: positive questionnaire or/and sweet taste ageusia at a sucrose concentration of 40 mg/l. Test 2 – specificity test: sweet taste ageusia at a sucrose concentration of 106.4 mg/l or/and salty taste ageusia at an NaCl concentration of 13.5 or 17 mg/ml. There were no inconclusive test results.

The study was approved by the proper Ethics Commission for Central Clinical Hospital of the Ministry of the Interior and Administration in Warsaw and was performed in accordance with good clinical practice and the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. No external funding or support was available.

A full gustatory function assessment (four flavours) was performed in all subjects. The tastant tablets were prepared in a manner analogous to the pilot study. The tastant tablet was a 0.33 ml gelatine capsule with added flavour. Each flavour capsule was placed in a separate package with a corresponding sample number. Every participant received 10 tablets – one tasteless reference and nine tastant tablets. Neither the patients nor the researchers knew the flavours of the individual samples. The patients were instructed to rinse their mouths with water and to begin testing the flavours with a tasteless sample that was clearly marked and subsequently proceed to the flavour samples (in a random sequence). After tasting each flavour, the participants were instructed to wait until the capsule had dissolved in the mouth and then describe the perceived taste on a form. No adverse events occurred during the test procedures.

Results

The characteristics of SARS-CoV-2 positive patients and SARS-CoV-2 negative students who served as a control group are listed in Table 1. The groups did not differ in terms of their age, height and weight. Almost all participants were male. There were no significant differences concerning the percentage of smokers or vapers in both groups. From the signs and symptoms characteristic of COVID-19 (listed in Table 1), only fever was significantly more frequent in the SARS-CoV-2 positive group, as well as self-reported smell and taste disorders.

Subjective smell or taste disturbances were reported by 65% of patients in the test group compared with 8% in the control group. After excluding the least specific testers (Sweet 1 and Sour 1; see Table 2) from the statistical analysis, taste disturbances were diagnosed in 50% of subjects in the test group and 22% in the control group.

Although a numerical difference was demonstrated between COVID-19 patients and the control group for all flavours, for a single flavour concentration, the difference was only significant with the Sweet 1 tastant tablet (p < 0.002). Detailed data are shown in Table 2.

The diagnostic values of the short medical questionnaire, including the subjective, self-reported loss of taste, self-reported loss of smell and fever in the last month, were calculated. These variables were chosen based on the significant difference of their frequency between the COVID-19 patients and the control group (see Table 1). The questionnaire was regarded as positive when at least one sign or symptom was present within the last 30 days. The sensitivity and specificity of the questionnaire in diagnosing COVID-19 are shown in Table 3. As can be seen, both the specificity and sensitivity were relatively high, but were not sufficiently high to create a reliable screening test (where the sensitivity should exceed 90%) or a diagnostic tool (with a specificity of over 90%).

For this reason, additional objective test measures were included to construct different prediction models. Adding the most sensitive tastant – Sweet 1 – to the medical questionnaire produced a model with a sensitivity of over 94% (Table 3 and Figure 2).

• Download figure
• Open in new tab

Figure 2.

ROC curves for prediction of the risk of a positive test set for SARS-CoV-2. The asymptotic significance of each model was less than 0.05.

However, no model, including the medical questionnaire, was sufficiently sensitive (over 90%). Therefore, a model was created using the most specific tastant tablets (Salty 1, Salty 2, Sweet 3). Although the tablets were not significant separately (Table 3), in parallel testing, the three tastants showed an extremely high specificity of 100%. The diagnostic test evaluation of the different test sets results is shown in Table 3. Figure 2 shows the ROC curves for the prediction models: questionnaire or/and Sweet 1 taste vs Salty 1 or/and Salty 2 or/and Sweet 3.

Discussion

Recently, taste and smell disturbances have been recognised as symptoms of COVID-19 and have even been listed as key diagnostic factors for COVID-19 by the BMJ.^17,18,19 Following this discovery, some institutions began to use questionnaires identifying loss of smell and taste as a screening for their visitors.²⁰ However, to date, to the best of our knowledge, no study has assessed the accuracy of taste disturbances as a screening tool for COVID-19.

This study confirms and demonstrates that chemosensory taste impairment can be regarded as a symptom of COVID-19. The reason for taste disturbances in COVID-19 is unclear; however, impairment of the renin–angiotensin system may be involved.²¹ In the present study, almost 70% of young, otherwise healthy SARS-COV-2 positive patients reported smell disorder and more than 60% reported taste disorder (Table 1). These figures are about 10 times higher than the matched control group, in which over 8% and almost 6% reported smell disorder and taste disorder, respectively.

This study is, to the best of our knowledge, the first to objectively assess COVID-19-associated taste disturbances and compare their frequency with a control group. It is also the first study to use varying flavour concentrations that were previously validated in healthy subjects. It was shown that the only taste that was significantly more frequently disturbed in SARS-CoV-2 positive patients was the sweet taste at the lowest flavour concentration (40 mg/ml) – a concentration that was perceptible to more than 90% of healthy subjects in a pilot study. This is a unique observation, particularly considering that, until now, reports regarding sweet taste disturbances in COVID-19 patients have been anecdotal.²²

Other researchers have identified a relatively similar frequency of taste disorders in COVID-19 patients. The highest prevalence of self-reported gustatory disorders identified is 88% in a multicentre European study. This figure is higher than the figure reported in the present research and may be due to the fact that the population described by Lechien et al. also included older subjects (up to 77 years) with concomitant diseases that can also affect taste and cause taste-threshold disturbances.5 No control group was included in the study by Lechien et al. In another study, Yan et al. found self-reported taste disturbances among 71% of SARS-CoV-2 patients and 17% of non-infected American subjects. These results seem to be comparable to ours. Although many of the control subjects in the study by Yan et al. exhibited influenza-like symptoms, including rhinitis and sore throat, which may have increased the frequency of smell and taste disturbances in the group,⁶ the same was true in our study (Tab. 1). Interestingly, the prevalence of dysgeusia reported for European and American populations appears to be much higher than in Chinese SARS-CoV-2 infected patients(5.6%), as reported by Mao et al,²³ and Korean patients (15.3%) as reported by Lee et al.²⁴

One study that objectively assessed gustatory dysfunction in COVID-19 patients only used single concentrations of particular flavors.¹¹ The patients in this study were considered to have ageusia if they were not able to identify any of the following four tastes: sweet, salty, sour and bitter. The frequency of ageusia according to this definition was 1.4% of study subjects, while 51.4% accurately identified all tastes. However, no control group was included in this study; in light of this, and due to differences in flavour preparation and concentrations, it is difficult to compare the results with those of the present study (although the figure of 51.4% is similar to the 50% of patients in the present study who accurately identified all tastes after the exclusion of Sweet 1 and Sour 1).

We do not consider the objective taste measurement (even the disordered sweet taste at a concentration of 40 mg/ml, which had a sensitivity of 71%; see Table 3) sufficient as a screening test for COVID-19. However, when this test was combined with a questionnaire consisting of three questions (whether patients had smell or taste disorders or fever in the last 30 days), the sensitivity increased to 94%. This appears to be an entirely acceptable value, taking into account that, for example, the sensitivity of the commonly used PSA screening of prostate cancer may be as low as 33% for a cut-off level of 3 ng/ml, sensitivity that makes this screening not reliable.^25,26 Our predictors appear to be similar to those found by Menni et al. in a large population sample, although these authors only utilised self-reported data, and the sensitivity and specificity (66% and 78% in a UK population, respectively) of the proposed model was much lower than ours.²⁷

A 94% sensitivity test, particularly if simple, inexpensive and safe, may be used for disease screening in large populations. Such a test may be also used repeatedly (e.g., once a week) to ensure fast identification of at-risk subjects and could be performed as a point-of-care test or as a self-administered test. Although we did not compare the results of the self-administered vs operator-administered test, a comparison with another, more complicated test showed no significant differences between the two procedures.²⁸ Such a test could also be used in developing countries in which the availability of genetic (RT-PCR for SARS-CoV-2) or serological (antibodies) screening is low. It could be used also in large companies or plants, where the need for nasopharyngeal swabs or blood draw, particularly when performed frequently, as well as the requirement to transport samples, may represent a significant burden.

The specificity of the proposed test is 55% (Table 3), which means that around as many as 50% of identified subjects will have COVID-19. This appears to be acceptable for a screening test, though the value is too low for it to be used as a reliable diagnostic tool. However, a test that combines three tastants (Salty 1 or Salty 2 and/or Sweet 3) demonstrated 100% specificity in diagnosing COVID-19. A positive result of such a test in asymptomatic or oligosymptomatic patients could serve as a basis for referring patients to quarantine in the event of limited availability of genetic or serological tests, as is the case in many countries in the world. It should be remembered that an estimated 50–70% of active cases may be asymptomatic at the time of diagnosis.¹⁵

This study has some limitations. A relatively low number of patients implies a validation in the bigger population, which is planned in the future. However, one of the strengths of this study is that it was performed on a uniform, young and otherwise healthy population living in a similar environment, with a comparable examined and control group.

The fact that the majority of subjects in the examined and control group in this study were men can be regarded as a limitation. However, as SARS-CoV-2 positive men seem to have gustatory dysfunction more rarely that women, the results could be even more reliable in women and in the general population.⁵ Another limitation is the limited accuracy of RT-PCR tests, which are the gold standard of COVID-19 diagnosis. To confirm the results of a PCR test, each patient underwent SARS-CoV-2 tests twice. In one case, a patient who initially received a negative diagnosis subsequently obtained a confirmed positive result.

In summary, the pandemic situation not only checks the state’s health security system but above all affects the sense of security of the individual. Regardless of psychophysical resilience, it destabilizes functioning, especially in the absence of information regarding potential infection. Hence, tests, especially those widely available, are crucial in satisfying the basic human needs related to their health safety.

At present, the gold standard for the detection of SARS-CoV-2 infection, according to the WHO, is RT-PCR examination of nasopharyngeal swab. However, the sensitivity of RTPCR as a screening method can be reduced in the case of incorrectly performed swabs. The costs of this method as a screening test is extremely high, the logistics involved in providing samples to reference laboratories is complicated, and the waiting time for results can be very long. Thoracic CT scan, on the other hand, demonstrates high sensitivity and specificity for COVID-19. However, in the context of screening, Thoracic CT scan also carries a number of disadvantages, including a high x-ray load during the procedure. Both methods are not accessible to the wider population in numerous countries.

As a SARS-CoV-2 vaccine is not yet available, and with only one vaccine candidate currently in a phase 2 trial, the only method that can currently control the pandemic involves testing the broader population for symptomatic, oligosymptomatic and asymptomatic carriers of SARS-CoV-2, isolating infected subjects and initiating early treatment of patients who develop COVID-19 symptoms.²⁹ An inexpensive, simple, fast and sensitive screening method is needed for this purpose. There is currently a public debate on the introduction of this type of test.³⁰ Considerable attention was paid to population testing strategies in the latest edition of the Lancet Infectious Diseases.³¹ Before sample-in-answer-out systems are introduced on a large scale, particularly in public utilities, and mass screening is introduced in developing countries, a system based on flavour tests could improve COVID-19 screening by identifying possible carriers of SARS-CoV-2 for further serological, molecular or other testing. Such a test, of the kind proposed in this study, could also be used to diagnose SARS-CoV-2 infection and serve as a basis for referring patients to quarantine in the event of limited availability of genetic or serological tests.