Screening COVID-19 by Swaasa AI Platform using cough sounds: A cross-sectional study

Sample size estimation and Data Collection

According to the sample size calculation, a total of 1152 comprising 40% COVID-19 positive cases and 60% control subjects was appropriate for validating if the device could detect COVID-19 with a 90% sensitivity on considering a 2.5% error for a 95% confidence interval (CI) and a prevalence of 0.75%. Considering all the conditions, we pooled the data collected from four individual clinical trial studies for developing and evaluating our model. We considered a total of 1052 participants in the present study, out of which 62% were controls. Control subjects comprise healthy individuals as well as subjects who were displaying various respiratory disease symptoms and came out positive for conditions such as asthma, Chronic Obstructive Pulmonary Disease (COPD), Interstitial lung disease (ILD), pneumonia but were negative for COVID-19 via RT-PCR.

The cough data was collected at Andhra Medical College, Visakhapatnam, India as a part of individual studies entitled “Development, Validation, Pilot Deployment of an ultra-scalable technology - Swaasa AI, as an auxiliary to COVID-19 Rapid test”, “COVID-19 Cough Sound Analysis Using Swaasa Artificial Intelligence Platform”. The studies were registered under Clinical Trials Registry-India CTRI/2021/09/036489, CTRI/2021/07/035096, and were begun after getting the approval from the AMC-Institutional Ethics Committee (IEC). The methodologies performed throughout the study were in accordance with the set guidelines. A duly signed written informed consent was also collected from all the enrolled subjects before starting the trial, which was followed by collection of the demographic details and the vital signs. Patients were then interviewed for the Part I of the St. George’s Respiratory Questionnaire (SGRQ) and COVID-19 symptoms in order to gather their symptoms [24]. Next, the cough sound was collected by a trained health care personnel. Each subject recorded multiple coughs (3-4 times), taking a breath in between every 15 seconds record interval. Following the cough sample collection, the patients were subjected to a reference standard test (RT-PCR).

The inclusion criteria for the enrolment are that the Patients must be (a) male and female patients age ≥ 18 years, who were (b) recently diagnosed with COVID 19 (for validation phase only) and were (c) able to read, understand and sign the informed consent form. Whereas male and female patients age < 18 years, who were (b) on ventilators support, (c) asymptomatic patients attending isolation ward for COVID testing and (d) Pregnant females were completely excluded from the current study. COVID precautionary and infection control measures were followed strictly.

Study design and Patient recruitment

The cross-sectional studies were undertaken in three phases. In the derivation phase (Phase 1) multiple data points (cough recordings) were collected from every subject to develop a robust model. Whereas, in the Phase 2 (validation) and Phase 3 (pilot test) only one data point (one cough recording) was collected.

Phase 1: Derivation with an objective to quantify the technical / analytical performance of the device by establishing a unique cough signature for COVID-19.

Sample size: For derivation phase, a total of 642 subjects were considered, which were part of two separate derivation studies. Out of which 252 subjects were tested positive for COVID-19 by RT-PCR. Figure 1 shows the age and gender distribution of the recruited subjects.

• Download figure
• Open in new tab

Figure 1:

Data distribution in the derivation phase, validation phase and pilot testing.

Phase 2: Clinical validation to quantify the performance of the device against clinical diagnosis based on reference standard test.

Sample size: For validation, 234 subjects were recruited from various locations including, isolation ward, COVID testing centre-King George Hospital, Visakhapatnam, COVID testing centre-GHCCD, Visakhapatnam and were subjected to the screening test using Swaasa AI Platform, as well as to the standard reference testing for diagnosis of COVID-19 i.e., RT-PCR. We compared the results obtained from the Swaasa AI Platform to the standard diagnostic testing.

Phase 3: Pilot test to quantify the effectiveness of the device when deployed as a screening tool prior to diagnosis.

Sample size: In pilot deployment, we enrolled 183 presumptive COVID-19 cases from a peripheral health care centre, RHC Simhachalam.

Data analysis

For phase 1 and phase 2, Swaasa’s performance was compared with diagnosis based on RT-PCR test. A consolidated test summary sheet was generated, which contained the results obtained from the classical gold standard diagnosis methods along with the Swaasa output. A statistician then compared both the results. The results obtained from the Swaasa AI platform were not accessible to the Physicians at any stage. For phase 3 the effectiveness of Swaasa was measured using the ratio of patients truly diagnosed as positive to all those who had positive test results based on Swaasa.

Event Extraction

Moving windowed signal standard deviation technique was used for extracting the events from the collected cough records [25]. A cough/non-cough classifier was used to separate the cough recordings collected from 252 COVID-19 positive subjects during the derivation phase into true cough and non-cough events like speech, silence, fan sounds, vehicle horns, and noise. A total of 1946 cough events were extracted at this step from both COVID positive and negative subjects.

Feature Extraction

Both the temporal and frequency domains of each cough event were used to extract the features. Zero crossing rate (ZCR) and energy are the two critical time domain features that were considered. Mel-frequency cepstrum (MFC), Spectral centroid, Spectral bandwidth, and Spectral roll off were the frequency domain features that were used for the data analysis [22].

The features were extracted from each frame of the cough signal. The average frame time was typically 20 milliseconds long. The coughing period can last anywhere from 200 to 700 milliseconds.

Prediction Model

Coughs were converted to Mel-frequency cepstral coefficient (MFCC) spectrograms to construct a multimodal architecture based on Convolutional Neural Network (CNN) and tabular features. Spectrogram images were fed as input to a CNN classifier for classification. At the same time, a tabular model was trained using features extracted from the frequency and time domains. Feature selection techniques have been used to remove redundant features and to identify the key features.

The total features extracted were 209, that includes age, gender, 120 MFCC (40 MFCC, 40 first derivatives of MFCC, 40 second order derivatives of MFCC), 9 spectral features (spectral centroid, spectral roll-off, spectral bandwidth, dominant frequency, spectral skewness, spectral kurtosis, spectral crest, spectral spread and spectral entropy), 33 chroma features (11 chroma, 11 first derivatives of chroma, 11 second derivatives of chroma), 18 contrast features (6 contrast, 6 first derivatives of contrast, 6 second derivatives of contrast), 15 tonnentz features (5 tonnentz, 5 first derivatives of tonnentz, 5 second derivatives of tonnentz), 3 Zero-crossing rate (ZCR, first derivatives of ZCR, second derivatives of ZCR), 3 Energy (Energy, first derivatives of energy, second derivatives of energy), 3 skewness (skewness, first derivatives of skewness, second derivatives of skewness), 3 kurtosis (kurtosis, first derivatives of kurtosis, second derivatives of kurtosis). We performed correlation analysis and recursive feature elimination (RFE) on these traits to rank them according to importance. As part of feature selection, we removed irrelevant features and reduced the number of features in the tabular model to 170.

The output of CNN and tabular models was merged to detect the presence of COVID-19 (yes/no/inconclusive). When the model is unsure if COVID-19 will be detected as yes/no, it provides an inconclusive output as shown in the block diagram Figure 2.

• Download figure
• Open in new tab

Figure 2:

Block Diagram illustrating the flow of the COVID-19 prediction model.

Spectrograms were given as input to a CNN (convolutional neural network) that was pretrained (using transfer learning (Resnet34) with Imagenet for training. In parallel, a tabular model with primary and secondary characteristics was trained. From each of these two models, the last fully connected layer was removed, combined with a new fully connected layer (connected layer), again passed through the linear layer, the activation layer of the final output layer. We call this approach of combining the last layers of two models as combined logic.

LIME Representation

The green portion of the Local interpretable model-agnostic explanations (LIME) representation [27] illustrates instances in which the model responded positively to a given class, whereas the red portion highlights instances in which it responded negatively. To “explain” a prediction, we refer to the display of textual or visual artefacts that give qualitative understanding of the link between the instance’s components (such as words in text, patches in a picture, etc.) and the prediction made by the model.

Statistical significance

A comprehensive assessment of model performance on the test set includes accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and ROC. To measure the variability of these parameters, we used the Clopper-Pearson method [28] with 95% confidence intervals. To better understand the model’s performance in screening COVID-19 subjects, we also calculated the confusion matrix across the test set.

Performance parameters in Model derivation phase

Cough sound data was collected from 252 COVID-19 positive subjects in the derivation phase. Data collected from 390 COVID-19 negative subjects in one of our earlier studies was also considered in this phase. Among 252 subjects, 60% were male and 40% were female, with age ranging from 18 years to 64 & above. Subjects were confirmed with COVID-19 by standard diagnosis methods. In this phase multiple data points were collected from the subjects. Each data point was called a record. A total of 803 cough records were collected from 252 patients.

The 252 subject data was divided into training (173) and test (79). The training data was internally divided into training and validation as required to build as well as optimize the model performance based on K-fold cross validation technique. All the 252 subject data was annotated with disease condition as COVID-19 i.e., COVID-19 likely as “yes”. For COVID unlikely, data representing other disease conditions was added from pre-existing datasets [29] (collected part of earlier studies) in various propositions. A total of 1213 records data was added to various classifiers. The final confusion matrix for derivation phase is represented in Table 1. The performance parameters such as accuracy, sensitivity, specificity, and AUC (area under the curve) of the model in the derivation phase are enlisted in Table 2.

The model was also evaluated on crowdsourced data, which includes COVID-19 Likely “yes” data from EPFL and Cambridge datasets, whereas COVID-19 Likely “no” is the in-house data. The final confusion matrix for this dataset is presented in Table 3. The performance parameters for the same are listed in Table 4. In both the cases an AUC value of > 0.85 AUC has been achieved.

The features listed in Table 5 depicts the mean value of the features extracted from individual frames, where we have considered normal as well as respiratory diseases data other than COVID from our previous validation study conducted at Apollo Hospitals, Hyderabad

LIME data comparison

Extremely low spectral frequencies have been observed in conditions such as Normal and Pneumonia as compared to asthma, which has an intermediate spectral frequency. On the other hand, we found that the spectral components are very high in diseases in which mucus accumulates in the airways and fluid accumulates in the parenchyma region. High spectral content is the distinguishing feature of COVID-19 cough from other respiratory diseases.

Feature analysis studies of cough sounds have revealed that it could be utilized for distinguishing diseases. The cough duration and frequency distribution has been found to be unique in a specific respiratory disease, including COVID-19 [30, 31].

We compared the LIME maps of various respiratory diseases with COVID-19 which are enlisted in Table 6. It can be seen in the maps that each disease has a unique frequency distribution. Green patches were more dominant for COVID-19 in high frequency regions. Whereas LIME maps for normal subjects were reacting negatively even though it has some green patches present. Similarly, pneumonia maps were also present in the high frequency range but has a stronger predominance in the medium frequency range. Even for Asthma most of the dominant green patches were seen in the medium frequency region.

From the LIME map analysis, we can conclude that COVID-19 related cough has a unique signature. These key signatures are detected by features extracted from coughs that can be further identified and characterized by machine learning models.

Performance Parameters of Model in Validation Phase

Out of 234 subjects participated in the validation phase, 22 were found to be COVID-19 positive and 211 COVID-19 negative by standard diagnostic methods such as RT-PCR.

Results of 1 subject remained inconclusive, hence didn’t consider that datapoint. In this phase only one cough record was collected from each subject. Confusion matrix for validation phase of Swaasa model is illustrated in Table 7, where the row represents the actual label, and the column represents predicted label. An accuracy of 75.54% with 95.45% sensitivity and 73.46% specificity was achieved for the Validation phase (Table 8).

Model Output in the Pilot phase

A total of 183 patients were recruited for the pilot testing phage. Out of these 183 subjects Swaasa was able to identify 82 subjects as having a likely presence of SARS-CoV-2. Out of these 82 subjects, 58 truly turned out to be COVID-19 positive with a Positive predictive value (PPV) of 70.73%. The confusion matrix for this phase is enlisted in Table 9.

The screening of COVID-19 patients by Swaasa is time saving as compared to the currently used traditional procedures. Additionally, Swaasa does not need any trained professional; a community healthcare worker can also perform the screening. The technician did not need any specialised equipment or supplies. The only prerequisites to complete the exam are a smartphone and a stable internet connection.