Abstract
The recently launched high-throughput assays for the detection of antibodies against SARS-CoV-2 may change the diagnostic strategies for COVID-19. This study aimed at investigating the performance of three novel high-throughput assays on Abbott Architect, Roche Cobas, and DiaSorin Liaison platforms. In addition, we evaluated a rapid lateral flow test from Dynamiker Biotechnology. A panel consisting of 133 samples including 100 pre-pandemic samples, 20 samples from SARS-CoV-2 PCR positive individuals and 13 possible crossreactive samples were used. Sensitivity and specificity in this study were equivalent with manufacturers’ data from Abbott and Roche but differed from those reported from DiaSorin and Dynamiker Biotechnology. The results suggest that the assays from Abbott and Roche could be considered in clinical use for individual patients.
Introduction
Coronavirus disease 2019 (COVID-19) was first reported in Wuhan, Hubei province, China, in December 2019 [1]. The virus causing this disease has since been designated SARS-CoV-2 [2]. The clinical manifestations of COVID-19 ranges from mild, or no respiratory symptoms, to severe viral pneumonia with need of intensive care and ventilator support. Due to the rapid dissemination of the infection, the outbreak was declared a pandemic by the World Health Organization (WHO) on 11th March 2020 [3].
RT-PCR has been considered the gold standard for confirming COVID-19 in patients, using respiratory samples obtained with nasopharyngeal and pharyngeal swabs, or less frequently, samples from the lower respiratory tract. Several RT-PCR assays have been developed, but most of them still exhibit a low degree of automation and therefore require substantial laboratory equipment, reagents and expertise [4]. Rapid antigen tests for respiratory samples have been introduced as a diagnostic option. However, a number of these tests have low sensitivity, thus decreasing their usability [5, 6].
Tests using the detection of SARS-CoV-2 antibodies in blood samples were rapidly developed but have until recently been limited to point-of-care tests, i.e., rapid tests, based on the colloidal gold-labeled immunochromatography principle. These tests usually deliver results within 15-30 minutes, using whole blood, serum, or plasma samples. Some of the tests claim to detect IgM or IgG, while others also identify IgA or a mixture of different classes of antibodies. Differences in test design affect the sensitivity and specificity of the tests [7].
Recently, novel high-throughput SARS-CoV-2 immunoassays have been introduced. We compared the performance of three high-throughput automated SARS-CoV-2 antibody assays together with a rapid immunochromatographic test.
Material and Methods
Assays
Three commercially available high-throughput immunoassays and their corresponding platforms were used for the study: Elecsys Anti-SARS-CoV-2 electrochemiluminescence immunoassay (ECLIA) on the Cobas 8000 e801 (Roche Diagnostic Scandinavia AB, Solna, Sweden); SARS-CoV-2 IgG chemiluminescent microparticle immunoassay (CMIA) on the ARCHITECT i2000 (Abbott, Illinois, USA) and LIAISON SARS-CoV-2 S1/S2 IgG chemiluminescence immunoassay (CLIA) on the LIAISON XL (DiaSorin, Saluggia, Italy). All samples were also tested using a rapid immunochromatographic test, 2019-nCOV IgG/IgM Rapid Test (Dynamiker Technology Co., Tianjin, China).
All three automated systems are part of the routine operations in our laboratories and are as such subject to accepted quality assurance procedures. Calibration and controls were performed according to manufacturers’ instructions. The rapid immunochromatographic test was performed manually with visual interpretation of the test results.
The antigen in the Elecsys Anti-SARS-CoV-2 assay is a recombinant protein representing the nucleocapsid (N) of SARS-CoV-2. The test is CE mark approved and detects antibodies, including IgG, in serum or plasma collected using standard sampling tubes. The duration of the analysis is approximately 18 minutes and the results are presented as a cutoff index as well as reactive or non-reactive.
The Abbott SARS-CoV-2 IgG reagent identifies the presence of anti-SARS-CoV-2 nucleoprotein IgG antibodies in serum or plasma using the Abbott ARCHITECT chemiluminescent microparticle immunoassay (CMIA). Results are obtained after 29 minutes and are reported as an index with a cutoff, distinguishing negative results from positive. The assay is not CE mark approved.
The LIAISON SARS-CoV-2 S1/S2 is a CE mark approved reagent that detects IgG antibodies recognizing the spike glycoprotein of the coronavirus. As for the previous tests, the antigens are composed of recombinant proteins expressed in human cell lines. Samples can be collected in standard serum or plasma tubes, and the analysis takes 35 minutes. The results are expressed as IgG antibody concentrations in arbitrary units (AU/mL) and are graded as negative, equivocal, or positive.
The 2019-nCOV IgG/IgM Rapid Test recognizes both IgG and IgM antibodies that bind to SARS-CoV-2 proteins in both serum and plasma samples. The assay is CE mark approved and results are presented after approximately 10 minutes as visually detected bands across the assay paper strip. For this test any positive result, either for IgG, IgM or both, was considered as positive.
Sample collections
The specificity of each assay was analysed using a collection of pre-pandemic samples consisting of 100 serum samples submitted to the Microbiology department during 2018. In order to challenge the assays further, 13 serum samples with possible interferences from the same year were selected and analysed using all four assays. This collection included five samples obtained during pregnancy; two with confirmed antinuclear antibodies (ANA); two with rheumatoid factor (RF-IgM); two with IgM antibodies against cytomegalovirus (CMV); and two samples with Epstein-Barr virus (EBV) IgM antibodies.
Assay sensitivities were evaluated using 20 serum samples from 16 individuals that prior to serum sampling had tested RT-PCR positive for SARS-CoV-2 in nasopharyngeal and/or pharyngeal swabs. The interval between onset of COVID-19 symptoms to serum sample collection ranged from 18 to 52 days (median 38 days). RT-PCR was performed using primers and probes targeting either the envelope (E) and the polymerase (RdRP) genes of SARS-CoV-2 as described by Corman [8], or the nucleocapsid gene (N) of the virus (Abbott RealTime SARS-CoV-2 Assay, Abbott Molecular Inc., Illinois, USA).
All samples originated from an existing sample collection at the Microbiology department obtained after consent to deposit, store and use for research and development. Samples were fully anonymized before inclusion into the study and results could hence not be linked to individuals. As a consequence thereof, the study did not require approval from an ethics committee, according to the guidelines of the Swedish Ethical Review Agency.
Calculations
Overall percent agreement, sensitivity (percent positive agreement), and specificity (percent negative agreement) were calculated based on a contingency table according to EP12-A2 [9], using Microsoft Excel 2019 (Microsoft Corp. Redmond, WA, USA). Further data analysis, including linear regression analysis and descriptive statistics, was performed using GraphPad Prism, version 7.04 for Windows (GraphPad Software, La Jolla California USA, www.graphpad.com). Good linearity was defined as R2>0.98 [10].
Results
Sensitivity and specificity obtained from this study, as well as information from the manufacturers’ are presented in Table 1. Roche had the highest sensitivity and Abbott the highest specificity. When calculating the sensitivity and specificity for DiaSorin, two RT-PCR positive samples and two samples from the negative collection were excluded due to equivocal results. The overall percent agreement was 97.5% for Abbott, 98.3% for Roche, 94.0% for DiaSorin, and 94.2% for Dynamiker Biotechnology.
The negative sample collection resulted in median values of 0.09 COI (0.08-3.16), 0.04 S/C (0.01-0.6) and 5.7 AU/mL (1.9-32.6) for Roche, Abbott and DiaSorin, respectively (Figure 1). The panel consisting of potentially cross-reactive pre-pandemic samples mainly gave negative results. However, for Dynamiker Biotechnology one sample with RF-IgM was positive for both SARS-CoV-2 IgM and IgG. One sample obtained during pregnancy showed an equivocal result on the DiaSorin assay. In contrast, all these samples were negative on the Abbott and Roche assays.
Three RT-PCR positive samples gave a weak positive result with Roche, close to the cutoff (1.01 to 1.26, cutoff ≥1.0) (Figure 1). These samples were reported as negative by DiaSorin, Dynamiker Biotechnology and Abbott. However, the Abbott values clearly differed from the rest of the Abbott negative samples by being closer to the cutoff (0.81-1.06, cutoff ≥1.4).
In a linear regression analysis of logarithmised results from the RT-PCR positive samples for all three high-throughput assays, a low degree of correlation was found with R2-values of 0.87, 0.58 and 0.39 for the combinations Roche/Abbott, Roche/DiaSorin and Abbott/DiaSorin, respectively.
Discussion
COVID-19 patients have been reported to have an incubation time ranging from 2.2 to 11.5 days (median 5.1 days) [11]. Recent findings show that antibody response to SARS-CoV-2 can be detected with a median seroconversion time for total antibodies at 11 days, IgM at 12 days, and IgG at 14 days from onset of clinical symptoms. The seroconversion rate reached 100% for total antibodies and IgM around one month after disease onset [12]. Another study found that 100% of the COVID-19 patients tested positive for IgG antibodies after 19 days [13].
As the pandemic progresses, it will likely become increasingly important to not only confirm ongoing COVID-19, but also to demonstrate evidence of past infection with SARS-CoV-2. On a community level this could be used for assessing the progress of the pandemic and for guiding public health and control policies [14]. On an individual level a reliable method for proving past COVID-19 could potentially be important for allowing employees to remain at work in spite of mild respiratory symptoms. It could possibly even serve as a basis for a proposed “immunity passport” [4, 15]. Although the degree of immunity to SARS-CoV-2 is disputed, it is plausible that individuals undergoing COVID-19 will gain partial or temporary protection against new episodes [16]. Serology tests might also be used to identify donors of convalescent plasma which has been proposed for treatment of seriously ill COVID-19 patients [17].
All four assays in this study performed satisfactory to be used on a community level. To be used on an individual basis, only Roche and Abbott performed well, with Roche exhibiting a sensitivity of 100%, and Abbott a specificity of 100%. With sensitivities of 77.8% and 75%, respectively, both DiaSorin and Dynamiker Biotechnology clearly underperformed in comparison to the specifications stated by the manufacturers.
Setting the cutoff value for a qualitative assay is commonly a tradeoff between sensitivity and specificity. The distribution of results for Abbott and Roche suggest that both assays potentially could benefit from a cutoff adjustment (Figure 1). In this study, a cutoff value of 0.8 for Abbott would result in a sensitivity and specificity of 100%. Increasing the cutoff for Roche to 4 would result in 100% specificity and 80% sensitivity. For DiaSorin the results were overall more narrowly distributed, and a significant overlap was seen between positive samples and negative controls, making it difficult to propose an optimal cutoff. In order to obtain a specificity of 100% for DiaSorin, a reduction of sensitivity to 50% would have to be accepted.
The strengths of this study was that we were able to evaluate four assays using the same sample collection. A limitation is the relatively low number of samples from SARS-CoV-2 positive individuals. Despite the low number of samples, our results are comparable with the performance reported in the kit insert from Abbott and Roche, but not for the DiaSorin and Dynamiker Biotechnology assays (Table 1). Another limitation is that we were not able to further investigate possible interferences in the false positive samples. Moreover, in our study only serum samples were included and conclusions can not be made regarding the performance of the assays in plasma (Abbott, Roche, DiaSorin and Dynamiker Biotechnology) or whole blood (Dynamiker Biotechnology).
Conclusion
We conclude that the SARS-CoV-2 assays from Abbott and Roche perform well and could be considered for introduction into clinical use. Further studies, with a larger number of samples positive for SARS-CoV-2-antibodies are needed.
Data Availability
The data are available from the corresponding author on a reasonable request.
Disclosure statement
The authors declare that they have no conflicts of interest relevant to the manuscript submitted to The Scandinavian Journal of Clinical and Laboratory Investigation.
Data availability
The data are available from the corresponding author on a reasonable request.
Funding
None to report.
Acknowledgments
The expert technical assistance of Ola Forsell and Susanna Bergqvist at the Department of Clinical Chemistry and Transfusion Medicine, Växjö Central Hospital, and of Sanna Hjalmarsson, Christina Bojesson and Eline Boesen at the Department of Clinical Microbiology, Region Kronoberg, is greatly appreciated.