Abstract
Background Rapid identification of SARS-Cov-2 infected individuals is a cornerstone in strategies for the control of virus spread. The sensitivity of SARS-CoV-2 RNA detection by RT-PCR is similar in saliva and nasopharyngeal swab. Rapid molecular point-of-care tests in saliva could facilitate, broaden and speed up the diagnosis.
Objectives and methods We conducted a prospective study in two community COVID-19 screening centers to evaluate the performances of a CE-marked RT-LAMP assay (EasyCoV™) specifically designed for the detection of SARS-CoV2 RNA from fresh saliva samples, compared to nasopharyngeal RT-PCR (reference test), to saliva RT-PCR and to nasopharyngeal antigen testing.
Results Overall, 117 of the 1718 participants (7%) were tested positive with nasopharyngeal RT-PCR. Compared to nasopharyngeal RT-PCR, the sensitivity and specificity of the RT-LAMP assay in saliva were 34% (95%CI: 26-44) and 97% (95%CI: 96-98) respectively. The performance was similar in symptomatic and asymptomatic participants. The Ct values of nasopharyngeal RT-PCR were significantly lower in the 40 true positive subjects with saliva RT-LAMP (Ct 25.9) than in the 48 false negative subjects with saliva RT-LAMP (Ct 28.4) (p=0.028). Considering six alternate criteria for reference test, including saliva RT-PCR and nasopharyngeal antigen, the sensitivity of saliva RT-LAMP ranged between 27 and 44%.
Conclusion In the ambulatory setting, the detection of SARS-CoV-2 from crude saliva samples with the RT-LAMP assay had a lower sensitivity than nasopharyngeal RT-PCR, saliva RT-PCR and nasopharyngeal antigen testing.
Registration number NCT04578509
Funding Sources French Ministry of Health and the Assistance Publique-Hôpitaux de Paris Foundation.
Introduction
Coronavirus disease 2019 (COVID-19) pandemic has caused significant impact on the healthcare system and socioeconomic activity. Early diagnosis is critical for prompt actions on patient management, infection control, and public health control measures (1). Since transmission can occur from asymptomatic or pre-symptomatic patients, mass testing, together with rigorous contact tracing and isolation, has been recommended to control the pandemic (2–4). This strategy implies rapid and reliable testing methods. Although molecular detection of SARS-CoV-2 RNA in nasopharyngeal swabs is considered as the “gold standard” for identifying infected individuals(1,5), nasopharyngeal sampling requires specific sampling equipment and trained personnel and may be difficult in some patients. Mass RT-PCR testing is carried out in specialized laboratories and needs several hours before results release. Altogether, these constrains restrain access to massive testing, increase time-to-result and consequently delay isolation of contagious individuals (6).
Rapid antigen point-of-care (Ag) testing allows to overcome the drawback of RT-PCR time-to-result but still requires nasopharyngeal sampling. Sensitivity of Ag tests was estimated at 50-90% and specificity at 90-100% as compared to nasopharyngeal RT-PCR (7,8). Recently self-anterior nasal sampling has been tested to reduce patient discomfort and avoid requirements for nasopharyngeal swabbing (9,10).
Self-collected saliva is non-invasive and easy to collect and thus more suitable for mass-screening than nasopharyngeal sampling (11–14). Recent meta-analyses assessed performances of saliva RT-PCR tests for the diagnosis of COVID-19 (15–18) and we recently confirmed in a large prospective study the excellent sensitivity of saliva RT-PCR, as compared to nasopharyngeal RT-PCR, for the detection of SARS-CoV-2 in community screening centers (19).
The combination of saliva sampling with rapid point-of-care testing could facilitate screening and isolation of infected individuals. Rapid single use RT-PCR assays for SARS-CoV2 RNA detection are available but were validated mainly on nasopharyngeal samples (20,21) and rarely in saliva (22), require sophisticated equipment and remain expensive. Nucleic acid detections based on isothermal amplification, such as loop-mediated isothermal amplification (LAMP), are interesting approaches as they simplify the analytical process, reduce the cost and enable to speed up the diagnosis. The sensitivity of RT-LAMP directly from NPS samples varies from 65% to 87 % compared to RT-PCR. Few studies tested RT-LAMP on self-collected saliva without RNA extraction. Sensitivities ranged from 45 to 85%, results being better after RNA purification than from crude samples (23–26). No studies estimated the performances of RT-LAMP on saliva samples as point-of-care systems directly in screening centers.
We conducted a prospective study in two community COVID-19 screening centers to evaluate the performances of a CE-marked RT-LAMP assay specifically designed for the detection of SARS-CoV2 RNA from fresh saliva samples compared to nasopharyngeal RT-PCR, saliva RT-PCR and nasopharyngeal Ag tests.
Methods
Study population and procedures
All adults and children, either symptomatic or asymptomatic, referred to the two participating COVISAN centers, Paris, France, were eligible as described previously (19). In accordance with EasyCoV® assay manufacturer instructions of (SkillCell-Alcen, Jarry, France), performances of saliva RT-LAMP were estimated on saliva tested in screening centers, immediately after collection (< 5 minutes) or stored immediately at 4°C and then tested within a maximal 90 minutes interval after collection. In addition, patients should have a valid nasopharyngeal SARS-CoV-2 RT-PCR test. Eligible persons received oral and written detailed information, adapted to their age. Data on sociodemographics, past medical history, presence of symptoms, consumption of alcohol, coffee, food, smoking and teeth brushing in the hours before testing were collected. The NPS was sent to the APHP high throughput platform for RT-PCR as part of routine care (reference method). Participants were asked to self-collect saliva sample after salivating 30 second in their mouth. Saliva were tested directly in the screening center (see below) and then centralized for RT-PCR testing and frozen at -80°C within 24 hours.
Virology methods
Nasopharyngeal RT-PCR
NPS were centralized and processed according to the routine procedure ((19), appendix). Nucleic acid extraction was performed with MGIEasy Nucleic Acid Extraction Kit (MGI Tech Co, Shenzhen, China) on a MGISP-960 instrument (MGI Tech Co). SARS Cov-2 RNA amplification was done using TaqPath™ COVID 19 CE IVD RT PCR Kit (Thermo Fisher Scientific, Coutaboeuf, France). The technique provides results expressed as a cycle threshold (Ct) for each gene target (ORF1ab, N and S-genes). The cutoff value of RT-PCR cycle threshold (Ct) retained to distinguish high/significant and moderate/low SARS-coV2 loads with TaqPath™ COVID 19 CE IVD RT PCR Kit was 28 (27). Ct values equal or higher than 32 corresponded to low viral loads.
Saliva RT-PCR
Saliva samples were tested at the APHP high throughput platform with RT-PCR on MGI instrument as described previously ((19), appendix). A 300 µl aliquot of saliva was mixed with 300 µl of NucliSENS® lysis buffer (Biomerieux, Marcy l’Etoile, France). Nucleic acid extraction and SARS Cov-2 RNA amplification were performed with the same procedure used for nasopharyngeal RT-PCR.
Saliva RT-LAMP
The test EasyCov® (SkillCell-Alcen, Jarry, France) is a CE-marked extraction-free RT-LAMP test specifically developed for saliva samples as point of care (saliva POC-LAMP). Detection of SARS-CoV2 was carried out according manufacturer’s instructions (EasyCOV®, SkillCell) (appendix). The procedure includes a step of virus inactivation and lysis at 80 ° C for 10 minutes and a step of viral genome amplification at 65 ° C for 30 minutes. The two steps take place in the Easyvid® system. After amplification, a reagent sensitive to pH is added to reveal the amplification. The result is immediately read by visual observation. The color turns yellow for a sample positive for SARS-CoV2 RNA and remains orange for a sample negative for SARS-CoV2 RNA.
Nasopharyngeal rapid antigen test
Nasopharyngeal Ag testing was performed with Standard Q COVID-19 Ag test (SD Biosensor®, Chuncheongbuk-do, Republic of Korea). Standard Q COVID-19 Ag test is a chromatographic immunoassay for the detection of SARS-CoV-2 nucleocapsid (N) antigen. The result was read after 15 to 30 minutes according to instructions of the manufacturer.
Statistical Analysis
Sample size was calculated assuming that the sensitivity of the index tests was equal or superior to 60%. To allow sufficient precision (± 10%), 93 subjects with positive nasopharyngeal RT-PCR were needed in each of the two subgroups (symptomatic and asymptomatic participants). As preliminary results indicates that viral load were not different between symptomatic and asymptomatic patients, the scientific committee of the study, during a planned meeting on 16 December 2020, recommended to perform the analysis as soon as 93 subjects with positive nasopharyngeal RT-PCR were included, whether symptomatic or asymptomatic.
RT-PCR results were considered positive if at least one gene was detected. Analyses of tests results were carried out blind of the result of the others and of the participant’s clinical data. For RT-PCR technique, Ct values reported are those for the ORF1a gene, and if not amplified, of the N-gene for (and of S-gene if the N-gene was not amplified).
Quantitative data were expressed as median [interquartile range], and qualitative data as counts (percentages). Diagnostic accuracy of the index tests was evaluated by calculating sensitivity and specificity. Confidence intervals were calculated by the exact binomial method. Subgroups analyses were performed according to: i) the presence of symptoms on day of testing, ii) the Ct value of the nasopharyngeal RT-PCR, expressed as low (at least one of the 3 targets with Ct ≤ 28, i.e. high viral shedding), or high (all 3 targets with Ct > 28, i.e. low viral shedding), and iii) to the consumption of alcohol, coffee, food, and smoking or teeth brushing before sample collection.
Sensitivity analyses were performed considering 6 alternate criteria for positivity for the reference standard: i) ≥ 2 positive targets with nasopharyngeal RT-PCR, ii) ≥ 1 positive target with nasopharyngeal RT-PCR and at least one of the 3 targets with Ct < 32, iii) ≥ 1 positive target with saliva RT-PCR, iv) ≥ 1 positive target with either the nasopharyngeal or saliva RT-PCR, v) ≥ 1 positive target with either the nasopharyngeal or saliva RT-PCR and at least one of the 3 targets with Ct < 32, and vi) NPS antigen test.
Quantitative variables were compared with Wilcoxon’s test, with a significance level of 5%. The statistical analysis was performed using R software (http://cran.r-project.org/). Reporting of results followed the Standards for Reporting Diagnostic accuracy studies (STARD 2015) guideline (28).
Role of the funding sources
The funding sources had no role in the study’s design, conduct and reporting.
Institutional Review Board (IRB) approval
The IRB Ile-de France III approved the study protocol prior to data collection (approval number 3840-NI) and all subsequent amendments.
Results
Participants
Between November 4th 2020 and February 15th 2021, 1718 participants were enrolled with a nasopagryngeal sampling for RT-PCR and saliva sampling for RT-LAMP assay. Details of samples collected and tests performed for nasopharyngeal antigen assay and saliva RT-PCR are detailed in Figure 1. Median age of study participants was of 37 years [26-52] and 55% were females (Table 1). Indications for testing and clinical symptoms reported on day of inclusion are detailed in table 1. One to three symptoms were observed in 530/1712 (31%) participants.
Characteristics of study participants. Results are presented as N (%) or median [interquartile range].
Study flowchart.
SARS-CoV-2 positive results
Overall, 117/1718 (7%) tested positive on nasopharyngeal RT-PCR: 78/691 (11%) in symptomatic and 39/1027 (4%) in asymptomatic participants (Table 2). Detection rates were of 2%, 9% and 7% for saliva RT-LAMP, saliva RT-PCR and nasopharyngeal Ag test, respectively.
Number of positive samples according to the technical procedure: nasopharyngeal RT-PCR, saliva RT-LAMP and RT-PCR, and nasopharyngeal antigen test.
Performance of detection of SARS-CoV-2 infection
Diagnostic accuracy of the two methods on saliva and the nasopharyngeal Ag test are presented in Table 3. Compared to RT-PCR on NPS, the sensitivity of saliva RT-LAMP was 34% (95% Confidence Interval (95%CI): 26-44). The sensitivity of saliva RT-PCR was 93% (95%CI: 86-97) and those of nasopharyngeal Ag test was 85% (95%CI: 77-91). The sensitivity and specificity of saliva RT-LAMP were similar in symptomatic and asymptomatic participants. Sensitivity analyses of saliva RT-LAMP according to six references (Table 4) showed similar results to the main analysis. Sensitivities of saliva RT-LAMP ranged between 27 to 44%. Its sensitivity was 37% (95%CI: 28-47%) compared to nasopharyngeal antigen test, and 30% (95%CI: 23-38) compared to saliva RT-PCR. Saliva RT-LAMP sensitivity was 40% (95%CI: 28-53) for Ct values below or equal to 28 and of 26% (95%CI: 15-40%) for Ct values above 28. As displayed on Figure 2, Ct values of nasopharyngeal RT-PCR were significantly lower in the 40 true positive subjects with saliva RT-LAMP (25.9 [19.4-30.2]) than in the 48 false negative subjects with saliva RT-LAMP (28.4 [24.4-32.6], p=0.028), with nasopharyngeal RT-PCR as reference test.
Diagnostic accuracy of the saliva RT-LAMP and RT-PCR, and the nasopharyngeal antigen test as compared to the reference standard (nasopharyngeal RT-PCR, positivity defined as at least one target gene detected), according to the presence of symptoms in study participants.
Sensitivity analysis of diagnostic accuracy of the saliva RT-LAMP test, as compared to the several references.
Nasopharyngeal SARS-CoV-2 RT-PCR
Ct values according to saliva RT-LAMP results. Ct values of SARS-CoV-2 RT-PCR on nasopharyngeal samples (NPS) in individuals with saliva samples negative with the RT-LAMP assay (False negative) and those with saliva samples positive with the RT-LAMP assay (True positive) are presented in boxplots.
Discussion
In this large prospective controlled study, the performance of a rapid RT-LAMP assay performed with crude saliva samples directly after saliva collection was analyzed. We used a CE marked assay specifically designed for saliva samples and for a point-of-care use. The test was authorized in France on November 2020 in symptomatic individuals for whom nasopharyngeal sampling was impossible or difficult. Our results showed, in a real-life rigorous evaluation, a low sensitivity of this method (34%) compared to nasopharyngeal RT-PCR. Its sensitivity remained low whatever the reference test considered (saliva RT-PCR, nasopharyngeal Ag test), ranging between 28 to 37%.
Our results differed strongly with the sensitivity of 86% (95CI 78%-94%) reported in Santos Schneider et al. study (29), and whatever the reference test used, nasopharyngeal RT-PCR or any other composite reference test including saliva RT-PCR and antigen test. In Santos Schneider et al. study, the authors evaluated the EasyCOV® assay in a central laboratory and tested each sample in triplicate. A sample was considered positive if at least two replicates out of three were positive. In our study, we tested all samples once directly in screening centers and according to manufacturer instructions, and to its expected use in routine conditions.
No difference in RT-LAMP sensitivity between symptomatic or asymptomatic participants was reported. Median time of testing was 3 days after symptom onset or 6 days after last contact of confirmed case. In 103 subjects already diagnosed for COVID-19, Nagura-Ikeda et al. reported, with another RT-LAMP assay, sensitivity results on saliva that differed according to the clinical state (23). The RT-LAMP assay was performed with nucleic acid extract of saliva instead of crude saliva. Overall sensitivity was of 71% compared to nasopharyngeal RT-PCR, with a higher sensitivity (85%) in patients tested within 9 days after symptom onset than after 9 days (44%). In asymptomatic individuals, the sensitivity was 60%.
According to other studies evaluating RT-LAMP tests, the critical step for sensitivity seemed to be the RNA extraction (13,23,25,30). A high level of concordance between RT-PCR on nasopharyngeal samples and RT-LAMP on saliva was observed when an automated extraction step (i.e. Qiasymphony RNA kit) was used. In a limited series of 34 positive samples (17 nasopharyngeal swabs and 17 saliva) tested by RT-PCR, Taki et al. reported a sensitivity of a RT-fluorescence LAMP assay performed with nucleic acid extracts of 97% and 100% in nasopharyngeal and saliva samples, respectively (25). Without RNA extraction on the same samples sensitivities decreased respectively to 71% and 47%, suggesting that RNA extraction process may be critical for the SARS-CoV-2 RNA detection by RT-LAMP especially for saliva samples. In our study, the RT-LAMP assay is an extraction-free test based on a 10 minutes heating at 80°C for virus inactivation and viral RNA release. This quick step suitable for a point-of-care test might be not optimal for RT-LAMP reaction with saliva samples and results may depend on miscellaneous factors according to the quality of saliva (volume, pH, viscosity, food by-products). The participants did not drink, eat or smoke within 30 minutes before saliva sampling. In addition, we did not find any significant effect of cigarettes or alcohol consumption within 2 or 24 hours. Another hypothesis is the impact of viral load. As we showed, Ct values of nasopharyngeal RT-PCR were lower in RT-LAMP true positive samples than in RT-LAMP false negative samples (26 vs 28), suggesting an impact of viral load on saliva RT-LAMP efficacy. However when considering only high or significant SARS-CoV-2 loads in nasopharyngeal samples, saliva RT-LAMP sensitivity remained low. Thus, the viral load per se does not explain the weak performance of the assay.
Finally, our study confirmed, as previously (19), the good performance of saliva RT-PCR and nasopharyngeal antigen testing as reliable alternative strategies to detect SARS-CoV-2 in both symptomatic and asymptomatic individuals in the ambulatory setting. Further work is needed to optimize an assay combining collected saliva and rapid point-of-care isothermal detection of SARS-CoV-2 RNA.
Data Availability
Individual data will not be publicly available, because participants did not provide consent for it.
Acknowledgments
The authors thank the following members of the scientific committee for insightful comments on the study protocol and results: Bruno Lina, Anne-Geneviève Marcelin, Catherine Paugam-Burtz, Astrid Vabret, Etienne Voirin-Mathieu, Michel Vidaud, Claire Poyart and staff involved in data collection: Claire Kappel, Guiseppina d’Anna, Elisabeth Velin, Abdulai Jalloh, Caroline Du Song, Philippine Treluyer, Mathilde Bayle, Lucie Daveau, Marine Hellegouarch, Suzie Zhu, Pénélope Travailleur, Matthieu Kapry, Adeline Huet, Aurélien Gibaud, Emeline Hermel, Céline Goy, Louise Lavancier.
Appendix
Appendix
SARS-CoV-2 RT-PCR on nasopharyngeal samples
NPS were inactivated at 56°C for 30 minutes. Nucleic acid extraction was performed on 180 µL of NPS with MGIEasy Nucleic Acid Extraction Kit (MGI Tech Co, Shenzhen, China) on a MGISP-960 instrument (MGI Tech Co). SARS-CoV-2 RNA amplification was done using TaqPath™ COVID 19 CE IVD RT PCR Kit (Thermo Fisher Scientific, Coutaboeuf, France). This test is a multiplex real-time RT-PCR test intended for the qualitative detection of nucleic acid from SARS-CoV-2. The kit contains three primer/probe sets specific to three different SARS-CoV-2 genomic regions (ORF1ab, N and S-genes) and primers/probes for bacteriophage MS2 used as internal control of amplification and extraction. A minimum of one negative control and one positive control was use for each run. Dilution of inactivated SARS-CoV-2 cell-culture supernatant (provided by Virology laboratory, Hospices Civils de Lyon) was added once a day in each production line as for external control. The technique provides results expressed as a cycle threshold (Ct) for each gene target.
SARS-CoV-2 RT-PCR on saliva samples
Saliva aliquots, stored frozen at minus 80°C, were thawed, equilibrated to room-temperature and then homogenized with a vortex for five seconds, and the 300 µl was mixed with 300 µl of NucliSENS® lysis buffer (Biomerieux, Marcy l’Etoile, France) and then extracted with the same procedure used for the nasopharyngeal samples. Saliva nucleic acids extracts were tested with the same RT-PCR procedure than for SARS-CoV-2 RT-PCR on nasopharyngeal samples.
RT-LAMP assay on saliva samples
The test EasyCov® (SkillCell-Alcen, Jarry, France) is a CE-marked extraction-free RT-LAMP test specifically developed for saliva samples. Detection of SARS-CoV-2 was carried out according manufacturer’s instructions (EasyCOV®, SkillCell) (appendix). In each screening center, saliva samples were tested immediately after collection (<5 minute) or stored immediately at 4°C and then tested within a maximal 90 minutes interval after collection. Briefly, 200 μl of saliva was mixed with the pretreatment buffer in tube 1 placed into the Easyvid® system for automated inactivation and lysis step (heating at 80 ° C for 10 minutes). Tube 1 is then taken out of Easyvid® and left to stand for 1 minute at room temperature. Three microliters of pretreated saliva sample are introduced into tube 2 containing RT-LAMP reaction mix. Tube 2 is incubated at 65 ° C for 30 minutes in dedicated positions of the Easyvid® system for viral RNA amplification. Once the 30 minutes have elapsed, tube 2 is taken out of the Easyvid® system and left to stand for 1 minute at room temperature. One microliter of revelation reagent is introduced in tube 2. The result is immediately read by visual observation. The color turns yellow for a sample positive for SARS-CoV2 RNA and remains orange for a sample negative for SARS-CoV2 RNA.
Footnotes
Corrections in Table 2
