Abstract
Increasing the diagnostic capacity of COVID-19 (SARS-CoV-2 infection) is required to improve case detection, reduce COVID-19 expansion, and boost the world economy. Rapid antigen detection tests are cheaper and easier to implement, but their diagnostic performance has been questioned compared to RT-PCR. Here, we evaluate the performance of the Standard Q COVID-19 antigen test for diagnosing SARS-CoV-2 infection and predicting contagiousness compared to RT-PCR and viral culture, respectively. The antigen test was 100.0% specific but only 40.9% sensitive for diagnosing infection compared to RT-PCR. Interestingly, SARS-CoV-2 contagiousness is highly unlikely with a negative antigen test since it exhibited a negative predictive value of 99.9% than viral culture. Furthermore, a cycle threshold (Ct) value of 18.1 in RT-PCR was shown to be the one that best predicts contagiousness (AUC 97.6%). Thus, screening people with antigen testing is a good approach to prevent SARS-CoV-2 contagion and allow returning to daily activities.
Introduction
Coronavirus disease 2019 (COVID-19) caused by SARS-CoV-2 has resulted in a global health crisis that requires substantial efforts worldwide to increase the diagnosis capacity and improve detection of cases by using inexpensive, easy, and rapid testing (1). On the other hand, the economic re-opening requires a diagnostic test before returning to daily activities. Therefore, a negative result in a diagnostic test has become the entrance door to many countries or to other activities that involve some risk of transmission (2).
Reverse transcription-polymerase chain reaction (RT-PCR) has become the standard gold test for SARS-CoV-2 detection due to its high sensitivity and specificity (3). However, it has been observed that RT-PCR remains positive well beyond the period of contagiousness. Therefore, the return to daily activities is delayed, generating unnecessary restrictions for patients in the convalescent period. In these patients, viral RNA can be detected in low loads (∼Ct value > 24 in RT-PCR), and contagiousness is unlikely. For this reason, classical techniques such as viral isolation are used as a better predictor of viral infectivity. However, these methodologies are expensive, risky, and time-consuming.
In 2020 point-of-care (POC) tests, such as antigen detection, were developed and approved (4). The first antigen test approved for diagnosis use in Colombia was the Standard Q COVID-19 Ag test (SD Biosensor, Republic ok Korea), which is based on immunochromatography and detects the nucleocapsid (N) antigen of SARS-CoV-2 using monoclonal antibodies (5). Antigen detection tests have shown desirable diagnostic characteristics such as reasonable specificity, fast execution, and easy processing (5,6).
The sensitivity and specificity of diagnostic tests are essential parameters used to guide decision-making regarding diagnostic tests. However, antigen tests are considered inferior to RT-PCR because their sensitivity for detecting SARS-CoV-2 is lower (7,8), likely since the performance of antigen tests can be affected by the days of symptoms onset (DSO) and viral load during infection (9,10). Likewise, the usefulness of the antigen test for predicting contagiousness remains to be determined (11). Thus, we aimed to evaluate the performance of a rapid antigen detection test approved for use in Colombia using RT-PCR and viral culture as the standard gold tests for diagnosing infection and contagiousness, respectively.
Materials and Methods
Population
A sample size of at least 250 specimens with a ratio of positive: negative of 1:1 was calculated, anticipating a 97% specificity according to reported previously (12,13), a probability of type I error of 5%, and a precision of 3%. The inclusion criteria to participate in the study were: adults (≥18 years) with suspicion of SARS-CoV-2 infection either by clinical or epidemiological criteria (clinical symptoms or recent exposure to a confirmed case, respectively). We analyzed 306 nasopharyngeal samples from 282 ambulatory subjects (some were sampled more than once as part of their follow-up) recruited at the Grupo Inmunovirología, University of Antioquia, Medellin, Colombia, between September 2020 and January 2021. We excluded pediatric patients and individuals who were unwilling to provide their written consent. The study was designed and conducted following the Declaration of Helsinki and Colombia legislation (Ministry of Health resolution 008430 of 1993). It was approved by the Ethics Committee of the Universidad de Antioquia (Act. 004, 02-04-2020). After explaining the project and clarifying doubts about the research, all included subjects signed the informed consent. The collected biological material was encoded to ensure privacy.
Sample collection
Nasopharyngeal swabs collected on viral transport medium were obtained following the Centers for Disease Control and Prevention (CDC) recommendations (14). The nasopharyngeal samples were maintained at 4°C for 4-84 hours before processing. The epidemiological and demographic data were collected from each subject filling out the official form for reporting acute respiratory infection by SARS-CoV-2 (National Institute of Health, Colombia) (15).
Antigen test
We used the Standard Q COVID-19 Ag test (SD Biosensor, Republic of Korea), following the manufacturer’s recommendations (6). Briefly, 350μL of the viral transport medium were mixed for 1-10 minutes with the antigen test-lysing reagent. Then, 4 drops were added to the cassette, and after 30 minutes, the test was interpreted as positive, negative, or invalid. The antigen tests were evaluated independently by two laboratory technicians blinded to the RT-PCR and viral culture results. In case of discrepancy, the antigen test was repeated with the same sample.
Real-time RT-PCR
Viral RNA extraction was performed from a 300 μL viral transport medium using the column-based Quick-RNA Viral Kit (Zymo Research, Orange, CA) following the manufacturer’s instructions. SARS-CoV-2 viral RNA was detected using the Luna® Universal Probe One-Step RT-qPCR Kit (New England Biolabs, MA, USA). The reaction included 7μL of viral RNA, the oligos and probe for the E gene and the conditions reported in the Berlin real-time RT-PCR protocol v2 (16) with a thermal modification in reverse transcription (55°C for 18 minutes) and in alignment/extension step (60°C for 30 seconds), according to the One-Step RT-qPCR Kit manufacturer’s recommendations.
In addition, human RNAse P gene transcripts were detected as internal control and evaluation of the quality of the sample, as previously recommended (17). The RT-PCR reactions were carried out in a CFX-96 Biorad thermal cycler (Biorad, CA, USA). Tests were performed in parallel with a negative control (sample replaced by water) and a positive control (RNA from virus isolated at the University of Antioquia) (18).
Viral culture
Viral cultures were carried out in a biosafety level 3 laboratory (BSL-3). Approximately 100 μL of the viral transport medium was dissolved in 250 μL of DMEM culture medium supplemented with 2% fetal bovine serum and 1% Penicillin-Streptomycin. This solution was inoculated into 1 × 105 Vero E6 cells monolayers in 12-well plates and incubated for 90 min at 37° C with 5% CO2. Subsequently, the inoculum was removed, and 1,5 mL of DMEM culture medium supplemented with 2% fetal bovine serum and 1% Penicillin-Streptomycin was added. The culture was kept at 37°C with 5% CO2 for 5 days. The monolayer was observed daily for a cytopathic effect (CPE; observed as rounding and detachment of infected cells) indicative of SARS-CoV-2 infection. After CPE observation, the supernatants were harvested and stored at −80°C. For confirmation purposes, some samples with or without CPE were evaluated for the presence of SARS-CoV-2 by indirect immunofluorescence (18) or by qRT-PCR on cell supernatants; all of these samples showed a perfect correlation between the presence of CPE and detection of the virus in the cell culture (data not shown).
Statistical analysis
The Standard Q COVID-19 Ag test was evaluated using the sensitivity, specificity, predictive value, and likelihood ratio calculated using GrapdPad Prism (version 9, California, USA), R v4.1.0, and the Integrated Development Environment Rstudio. To explore variables associated with the result of each test, bivariate analyses were carried-out, and according to statistical (p-value <0.05) and plausibility criteria, they were included in a binary logistic regression model. Receiver Operating Characteristic (ROC) curves were constructed to discriminate the best cycle threshold (Ct) value predicting contagiousness.
Results
Demographic and clinical characteristics of the population
In this study, 306 nasopharyngeal samples were analyzed. All samples were blinded and codified before the analyzes. The median age of subjects was 38 years [Range 18 - 96], and 58.9% were female. Considering that the performance of diagnostic tests may fluctuate depending on DSO, we allocate the samples in four diagnostic sceneries: i) 108 samples came from subjects with 1-5 DSO, ii) 50 from individuals with 6-11 DSO, iii) 9 samples were from individuals with more than 11 DSO, and iv) 139 samples were from SARS-CoV-2-exposed subjects without symptoms (some symptomatic subjects also disclose previous close contact with a person diagnosed with COVID-19) (Figure 1).
figure shows the percentage of positive results obtained by RT-PCR, antigen test, and viral culture in patients who were on 1-5 days of symptom onset (DSO), 6-11 DSO, >11 DSO, and people who were asymptomatic individuals.
The percentage of positivity in RT-PCR was higher than 70% in individuals with symptoms but below 40% in asymptomatic ones. In contrast, less than 40% positivity by antigen test and viral culture were seen in subjects with either 1-5 or 6-11 DSO, and no positivity was observed in subjects with more than 11 DSO. As expected, the positivity in viral culture decreased as days with symptoms increased (Figure 1).
The definitive diagnosis of infection was made by RT-PCR. In total, 176 samples were SARS-CoV-2 RT-PCR-positive (57.5% positivity), most of them from outpatients who resolved COVID-19 at home, and seven were from hospitalized patients. The clinical and demographic characteristics of subjects in the study were recorded in search of associations with the result of antigen test (Table S1) and viral culture (Table S2).
Performance of antigen test in the diagnosis of infection
People who tested positive by antigen test were older (age median [range] = 45.5 [20.0-96.0] than people tested negative (age median [range] = 37 [18.0-84.0]), and a higher proportion of them exhibited cough, fever, odynophagia, dyspnea, fatigue, conjunctivitis, headache, and anosmia or ageusia, compared to those with a negative antigen test. No differences in gender between subjects with positive or negative antigen tests were observed (Table S1).
The performance of the SARS-CoV-2 antigen test for infection diagnosis was evaluated with RT-PCR as the reference standard. From 176 RT-PCR positive specimens, 72 were positive by antigen test, revealing a low sensitivity (40.9%, confidence interval (CI) 95%= 33.6-48.6%). No negative RT-PCR sample was positive in the antigen test, demonstrating high specificity for SARS-CoV-2 infection (100% CI 95% = 96.4%-100%). Indeed, the probability of COVID-19 was high in the participants with a positive result in the antigen test (positive predictive value, PPV 100%, CI 95%= 93.7-100%). Nonetheless, the concordance between the antigen test and RT-PCR was generally low, with a kappa index of 0.37 (CI 95%= 0.29-0.44).
We observed the best performance of antigen test in samples from patients with 1-5 DSO, in which the diagnostic sensitivity was 57.9% (CI 95%= 46.0%-68.9%), the specificity was 100% (CI 95%= 86.7%-100%), and concordance with the RT-PCR using the kappa index was 0.44 (CI95%= 0.32-0.58). In patients between 6 and 11 DSO the sensitivity and specificity of the antigen test were 52.4% (CI 95%= 36.6%-67.7%) and 100% (CI 95%= 59.8%-100%), respectively. In asymptomatic subjects, the performance of the antigen test decreased, showing a poor sensitivity (12.0% CI 95%= 5.0%-25.0%), although the specificity was similar to the other categories (100% CI 95%= 94.8-100%) (Table 1).
Positivity of Antigen test and viral culture is linked to Ct value in RT-PCR
Considering that the Ct value in RT-PCR is an indicator of viral load, we explored the distribution of Ct values between positive and negative samples by antigen test and viral culture (Figure 2). Interestingly, the Ct values among both antigen test and viral culture had a similar distribution, being lower in samples that tested positive (median Ct value [interquartile range] of 15.2 [12.1-18.1] and 13.7 [11.6-15.6] respectively) than in the negative samples (28.93 [16.7-36.6] and 27.47 [11.8-36.6] respectively, Figure 2). Indeed, out of 65 samples with a Ct<20 by RT-PCR, 62 (95.4%) were also positive by antigen test, and 51 (78.5%) were also positive by viral culture.
Figure displays the distribution according to of cycle threshold (Ct) value obtained in RT-PCR of positive and negative results of antigen test and viral culture in patients with SARS-CoV-2 infection.
Prediction of contagiousness
All samples were assessed by viral culture and antigen test, even when the RT-PCR was negative. SARS-CoV-2 was isolated in 53 (17,3%) of 306 samples (30% when considering only the 176 samples positive by RT-PCR). The clinical and demographic characteristics of population were recorded in search of associations with the result of viral culture, and thus its usefulness to predict contagiousness. People with a positive culture test were older (a median age [range]= 44.0 [20.0-96.0] than people without SARS-CoV-2 isolation (a median age [range]= 38.0 [18.0 – 86-0]), and had a significantly higher frequency of fever, cough, odynophagia, fatigue, dyspnea, headache and anosmia or ageusia, compared to those with a negative culture (Table S2).
We subsequently explored the usefulness of antigen tests in predicting contagiousness, as measured by viral isolation in vitro (Table 1). The concordance between viral culture and antigen test was acceptable, with a kappa index of 0.77 (CI95%= 0.68-0.85). Of 53 viral culture-positive samples, 51 were also positive in the antigen test, showing a high sensitivity (96.2% CI 95%= 85.9%-99.3%). In addition, we observed a high negative predictive value of 99.1% (CI 95%= 96.6%-99.9%) in predicting contagiousness (Table 1). Likewise, we found that the antigen test predicts contagiousness more accurately 1-5 DSO (kappa index of 0.82, CI95%= 0.71-0.93) than 6-11 DSO (kappa index of 0.44, CI95%= 0.21-0.67).
Asymptomatic subjects that were RT-PCR positive, usually have a low viral load (median Ct [interquartile range] = 29.6 [26.79-33.44]); therefore, most samples were negative by antigen test (95.6%) and viral culture (96.4%). Nevertheless, in this group, the sensitivity of the antigen test for contagiousness was high (100% [CI 95%= 46.29%-100%]) (Table 1).
We carried out a multiple logistic regression model (Table S3) with variables such as the Ct value in RT-PCR, and those showing a crude association with antigen test or viral culture results (Tables S1 and S2). Although positivity in antigen test was associated with positivity in viral culture [crude OR=134.79 (30.45,596.64)], the association was not statistically significant after covariates adjustment [adjusted OR=5.79 (0.5,66.85)] (Table S3). In contrast, the categorized Ct value was associated with the positivity in viral culture after adjustment. Indeed, the samples with a Ct<20 and <15 had adjusted ORs of 25.07 (2.27, 277.23) and 290.45 (17.19, 4907.16)] respectively (Table S3).
Predicting contagiousness by RT-PCR
According to the previous findings, the RT-PCR Ct value, as an indicator of viral load, could help predict infection. Therefore, we analyze the Ct value to find the value that best predicts a positive result in viral culture. An analysis of a receiver operating characteristic (ROC curve) was performed, obtaining a Ct value of 18.1 as the best predictor of viral culture result, with an area under the curve (AUC) of 97.6 % (95% CI = 95.6% - 99.5%) (Figure 3). Our results also showed that the highest Ct value where a viral isolate was obtained was 23.5 (Figure 2).
The Receiver operating characteristic (ROC) curve shows the Ct value where sensitivity and specificity are higher than 95% to predict the contagiousness using viral culture as reference. Area Under the Curve (AUC) 97.6% (95.6%-99.5%).
Discussion
We evaluated the Standard Q antigen test performance to diagnose SARS-CoV-2 infection and predict contagiousness. This rapid and simple antigen test is widely used in Colombia and other countries, especially in high prevalence populations; its sensitivity has varied widely across different evaluations (13,16,19–22). In this study, the sensitivity for infection diagnosis in all samples was low, 40.9% (CI 95%= 33.6,48.6%), much lower than the 84% sensitivity reported by the manufacturer (6). Nevertheless, other authors have reported significant variation in sensitivity ranging between 42.9% (23), 70.7% (24), and 98.3% (16).
This variability seems to be only partially explained by the DSO at sampling because the sensitivity of antigen test in subjects with 1-5 DSO increases up to 57%, whereas in subjects with 6-11 DSO the sensitivity was 52%. Indeed, other studies have reported similar sensitivity values in subjects with ≤14 DSO (55.4%) (22). On the other hand, our sampling population consists mostly of outpatients, which usually have mild disease and consequently lower viral load. Actually, we observed similar sensitivity compared to 63% reported in patients with the same disease spectrum (25). Indeed, when we evaluated the performance of the antigen test compared to the Ct in RT-PCR, the proportion of positive antigen test was 62/65 (95.4%) in samples with Ct<20, in concordance with other reports indicating around 95% sensitivity of antigen test in samples with high viral load (Ct less than 22.5) (26,27).
Although the sensitivity of the antigen test was low, its high specificity and high positive predictive value indicate that a positive result in this test provides reliable evidence of SARS-CoV-2 infection; conversely, a negative result does not rule out infection. However, additional testing in subjects with a negative antigen test could be unnecessary if we are interested in detecting patients who are shedding viruses and therefore are a threat to those close to them.
Remarkably, the antigen test and viral culture performance were similar; in both cases, the distribution of Ct values had a median <20 in positive results. Whereas there were no specific symptoms typical in COVID-19 or DSO related to the probability of viral isolation, the antigen test showed a specificity of 91% to predicts viral infectivity; from the 306 samples, only 2 were discordant between antigen test and viral culture results. In addition, our findings are consistent with studies such as Pekosz et al (28), who reported that the antigen tests correlate with viral culture of SARS-CoV-2 in 27/28 samples assessed. Yamayoshi et al (26), in samples with a Ct value <22.5 found that antigen test result was similar to viral isolation in culture (11/11 were positive in antigen test and 8/11 in viral culture). Therefore, these results suggest that an antigen test is a promising tool to discharge patients from quarantine because of the low probability of transmitting the virus.
Although viral culture is the most obvious method to assess contagiousness, it is complex, expensive, and requires particular biosafety conditions. The Ct value of RT-PCR, despite the usefulness in the prediction of contagiousness in samples with high viral load, is intrinsically variable (25,29,30). The variation in Ct can be due to the conditions of the RT-PCR reaction, the fluorescence threshold, the fluorochrome used, the gene target, and the virus lineage (10). Antigen test, which targets the more conserved nucleocapsid protein, is potentially a more consistent and cost/benefit method of assessing contagiousness.
Although some authors have reported viral isolation in culture up to Ct value close to 32 (25,31,32), most studies have associated viral isolation with Ct 18-24 (5.4 – 7.0 log RNA copies) (33,34). Our study indicates that the Ct value that best predicts the result in viral culture is 18.1, with a progressive decrease in the possibility of contagiousness above this Ct value. Hence, we propose that a Ct value higher than 23.5 could be a safe threshold of contagiousness since it was the highest Ct in which we could isolate the virus in culture.
In this study, the time of exposure was not controlled since it was reported by each individual, considered from the last contact with someone with a COVID-19 confirmed diagnosis. Due to this, many individuals did not know precisely the time of exposure to the virus. Additionally, the data correspond to a study of diagnostic tests in which the prevalence of infection is given by the proportion of infected and uninfected subjects who participated in the study, which could not be an exact reflection of the actual prevalence in the population. Further studies should include a serial sampling of the participants in other prevalence scenarios to improve the scope of the results.
Conclusion
This study demonstrated that the Standard Q COVID-19 antigen test has excellent agreement with viral culture, indicating that it can be used as a marker of contagiousness. Due to its high positive predictive value in situations of a high prevalence of infection, positive results do not require confirmation with another test. Likewise, its high negative predictive value for contagiousness makes it possible to use this test as a criterion to discharge patients in isolation and screen people moving into environments that could facilitate the transmission of the virus. Additionally, we found that 18.1 is the Ct value in RT-PCR that best predicts the possibility of in vitro viral isolation, related to a higher probability of contagion and viral transmission.
Funding
This research was funded by Universidad de Antioquia (UdeA responde al COVID-19) and Grupo Inmunovirología, Facultad de Medicina, Universidad de Antioquia.
Author Contributions
“Conceptualization, T.J.L, F.J.D, J.C.A, M.T.R, and W.A.J.; methodology, T.J.L, and W.A.J.; formal analysis, J.C.A, T.J.L, and W.A.J; writing—original draft preparation, T.J.L, J.C.A, and W.A.J; writing—review and editing, M.T.R, J.C.A., F.J.D., and W.A.J. All authors have read and agreed to the published version of the manuscript.”
Acknowledgments
The authors thank the patients and volunteers who kindly participated in this study. We also thank the clinicians who support the recruitment of patients and laboratory assistants who contributed to the sampling.