ABSTRACT
The main objective of this study was to estimate the performance, under local epidemiological conditions, of two in-house ELISA assays for the combined detection of anti-SARS-CoV-2 IgA, IgM, and IgG immunoglobulins. A total of 94 serum samples were used for the assessment, where 44 corresponded to sera collected before the pandemic (free SARS-CoV-2 antibodies), and 50 sera were collected from confirmed COVID-19 patients admitted to the main public hospital in the city of Valdivia, southern Chile. The Np and RBD proteins were separately used as antigens (Np and RBD ELISA, respectively) to assess their diagnostic performance. A receiver operating characteristic (ROC) analysis was performed to estimate the optical density (OD) cut-off that maximized the sensitivity (Se) and specificity (Sp) of the ELISA assays. Np ELISA had a mean Se of 94% (95% CI = 83.5 – 98.8%) and a mean Sp of 100% (95% CI = 92.0 – 100%), with an OD 450 nm positive cut-off value of 0.88. On the other hand, RBD ELISA presented a mean Se of 96% (95% CI = 86.3 – 99.5%) and a mean Sp of 90% (95% CI = 78.3 – 97.5%), with an OD 450 nm positive cut off value of 0.996. Non-significant differences were observed between the Se distributions of Np and RBD ELISAs, but the latter presented a significant lower Sp than Np ELISA. In parallel, collected sera were also analyzed using a commercial lateral flow chromatographic immunoassay (LFCI), to compare the performance of the in-house ELISA assays against a commercial test. The LFCI had a mean sensitivity of 94% (95% CI = 87.4 – 100%) and a mean specificity of 100% (95% CI = 100 – 100%). When compared to Np ELISA, non-significant differences were observed on the performance distributions. Conversely, RBD ELISA had a significant lower Sp than the LFCI. Although, Np ELISA presented a similar performance than the commercial test, this was 2.5 times cheaper than the LFCI assay. Thus, the in-house Np ELISA could be a suitable alternative tool, in resource limited environments, for the surveillance of SARS-CoV-2 infection, supporting further epidemiological studies.
INTRODUCTION
The emergence of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the etiological agent of COVID-19, has caused a global pandemic, which 2.5 years after the first reported case (December 2019) has been linked to more than 5 million deaths and more than two hundred million cumulative cases [1]. Since its emergence, several serological assays have been developed to detect the presence of specific antibodies against SARS-CoV-2 [2]. Although, serological approaches cannot distinguish between acute and chronic infection, these types of tests are useful for i) the identification of individuals who have developed an immune response, ii) aid in contact tracing, iii) monitoring infection dynamic in the general population, and iv) the development of clinical trials [3,4].
The immunoglobulin time response is between 4 to 10-14 days after the onset of symptoms, which limits its applicability for the diagnosis during the acute phase [5,6]. However, the IgM and IgA anti SARS-CoV-2 antibodies develop rapidly in response to the infection, and their detection can significantly increase the diagnostic sensitivity of SARS-CoV-2 infection, when serological tests are combined with molecular tests [7]. In particular, IgA antibodies play an important role in mucosal immunity, where IgA may be a better marker of early infection than IgM [8–10]. However, most COVID-19 serological tests are based on the detection of IgM and/or IgG antibodies [11]. The preference of IgG and/or IgM detection over IgA, probably is related to a lower specificity of this immunoglobin despite an earlier onset in comparison with IgG and IgM [9]. Nevertheless, the use of assays detecting IgA, along with IgG and IgM, may be useful in scenarios where it is necessary to maximize the diagnostic sensitivity of the test, such as a screening tool for a surveillance program, for example. Additionally, IgA assays may also be helpful in patients with atypical symptoms, in asymptomatic cases, or when RT-qPCR results remains negative in suspected subjects [12,13].
The detection of circulating antibodies against SARS-CoV-2, as part of a surveillance program, requires the use of tools with known sensitivity and specificity [14]. Those parameters would allow to estimate key epidemiological variables, such as the true prevalence (TP), when the assay is used in randomized studies. Moreover, accurate TP estimates could be used to assess the performance of passive surveillance systems that most countries have implemented as part of their COVID-19 control strategies [15,16]. The main serologic assays used for SARS-CoV-2 detection include the lateral flow chromatographic immunoassay (LFCI) and ELISA tests. In these kinds of assays, the most used viral proteins as antigens are the nucleocapsid protein (Np), which plays a role in the transcription and replication of the virus [17,18] and the subunits S1 and S2 of the spike (S) protein [19]. Specifically, S1, containing the receptor binding domain (RBD) for the host angiotensin-converting enzyme (ACE2) receptor; and the S2, containing elements needed for membrane fusion [20,21]. Previous evidence has suggested that the IgG antibodies aiming for the S protein are more specific than the anti-Np protein [22,23]. On the other hand, the IgG aiming Np may be more sensitive than those anti-S proteins, particularly in the early phase of infection [22,23]. This could be explained by the relatively high homology in aminoacidic sequence of the SARS-CoV-2 protein Np with the nucleocapsid proteins of other Coronaviridae and other viruses [22,24]. Thus, the increased sensitivity of the anti-Np antibody response detection could be at the expense of specificity. The latter may be due to the potential cross-reaction of serological tests to other similar viruses circulating in the target population, increasing the false positive rate. In consequence, the performance of any serological assay, such as ELISA tests must be optimized and validated under local conditions, accounting for endemically circulating viruses.
The objective of this study was to evaluate and validate two in-house ELISA assays for the combined detection of SARS-CoV-2 IgG, IgM and IgA antibodies. To be used for the surveillance of COVID-19 in the general population, and to support further epidemiological studies. In particular, diagnostic performance indices, such as sensitivity (Se) and specificity (Sp), were estimated using sera from pre-pandemic individuals and confirmed COVID-19 patients
MATERIAL AND METHODS
Study population, sample collection and ethics approvals
The present study was developed following the Standards for the Reporting of Diagnostic Accuracy Studies (STARD) guidelines proposed by Cohen et al. [25]. A total of 94 individuals were enrolled, where 50 corresponded to confirmed cases of SARS-CoV-2 infection (24 women and 26 men), and 44 corresponded to sera collected between June and July 2019, thus regarded as free of SARS-CoV-2 antibodies sera (pre-pandemic samples). All 50 confirmed cases corresponded to unvaccinated patients admitted to the Hospital Base de Valdivia (HBV), Valdivia, southern Chile, from April to November 2020, and they were confirmed by a standard RT-qPCR assay using nasopharyngeal swab samples, as previously described [26]. At the time of sera collection at the HBV, all were symptomatic cases, presenting different degrees of COVID-19 complications, from very mild to severe, which eventually required hospitalization at the HBV. In this line, confirmed cases presented a median of 4 days (interquartile range (IQR): 2 – 6 days) between symptoms onset and RT-qPCR diagnosis, whereas serum samples were collected with a median of 11.5 days (IQR: 9 – 15 days) after symptoms onset. This group presented a median age of 58 years (1st Q: 52.5 years and 3rd Q: 68.5 years). On the other hand, samples from non-COVID-19 individuals were obtained from a serum bank of an epidemiological study on cystic echinococcosis, where sera were collected from the general population in the Coquimbo region, Chile. This group was composed of 27 women and 17 men, with a median age of 53.5 years (1st Q: 41.5 years and 3rd Q: 63.5 years), and 7 out of 44 control-participants presented some type of chronic disease, mainly diabetes.
Serum samples from non-COVID-19 individuals were obtained and handled following the protocol accepted by the Scientific Ethics Committee of the Faculty of Medicine at the Universidad Católica del Norte, Coquimbo-Chile, approved under the resolution CECFAMED-UCN N º 81/2019. In short, 5 cc of blood was obtained by peripheral venipuncture from volunteers who attended an ultrasound survey for detecting cystic echinococcosis in urban and rural areas in northern Chile. Blood samples were then centrifuged on the same day using a portable centrifuge (Mobilspine, Vulcon Technologies, Richmond, USA), serum was separated and kept in liquid nitrogen in the field and then transferred to a −80ºC freezer until further analysis. The samples from the confirmed COVID-19 patients were collected following the sample collection protocol of the HBV, where 6 cc of blood were obtained by peripheral venipuncture using yellow or red cap tubes and processed before 4 hours. The use of stored sera, for COVID-19 research purposes, was authorized by the Scientific Ethics Committee of the Servicio de Salud Valdivia (SSV), Ministry of Health of Chile, under the resolution SSV Ord.N°187/2020.
Study design and laboratory analyzes
The in-house ELISA aims for the combined detection of specific anti-SARS-CoV-2 isotype antibodies (IgA, IgM, and IgG) for the surveillance of COVID-19. To assess the performance of this assay, sera from pre-pandemic and confirmed cases were assessed by the in-house ELISA test using Np and RBD proteins separately as antigens (Np ELISA and RBD ELISA, respectively), in order to evaluate the performance of each protein. Additionally, analyzes were run in in duplicate and reported optical density (OD) corresponded to the mean between runs. As a quality assurance element, the coefficient of variation between runs was estimated.
In-house ELISAs for combined detection of specific SARS-CoV-2 antibodies
Proteins: Recombinant SARS-CoV-2 Np protein (Met1-Ala419, with a C-terminal 6-His tag) from Spodoptera frugiperda (R&D System, Catalog Number 10474.CV) (accession # YP_009724397.2). Recombinant SARS-CoV-2 S1 subunit protein (Arg319-Phe541, with a C-terminal 6-His tag) of Host Cell Receptor Binding Domain (RBD) from HEK293 cells (Raybiotech, Catalog Number: 230-30162) (accession # QHD43416.1); Coating step: A final volume of 50 µl of 20 ng Np or RBD proteins [27] were seeded in a 96-well plate in carbonate buffer pH 9.6 per well and incubated overnight at 4°C. Washing step: After the incubation time, 3 washes of 5 minutes were carried out with PBS 1x-Tween20 0.05%. Blocking step: the wells were blocked with 200 µl of blocking solution (1x PBS, 5% BSA, 0.05% Tween20) for 2 hours at 37° C, and the content of the plate was discarded, and the washing phase was carried out as previously described. Loading of serum samples: A final volume of 50 microliters of sample (1:40 dilution in PBS buffer 0.1% w/v of BSA; 0.05% v/v Tween20) [27,28] was loaded and incubated for 2 hours at 37° C. Washing step: After the incubation time, 3 washes of 5 minutes were carried out with PBS 1x-Tween20 0.05%. Antibodies hybridization step: 100 µl per well of Anti-Human IgA/IgG/IgM (H&L) goat polyclonal antibody (HRP) (Rockland, R.609-103-130) at a dilution of 1: 10,000 in PBS buffer 0.1% BSA-0.05% Tween20, were added and incubated for 1 hour at 37° C, the content of the plate was discarded, and the washing phase was carried out as previously described. Finally, the washed wells were developed with TMB reaction and read at 450 nm using a microplate reader HR801 (Shenzhen Highcreation Technology Co. Ltd).
Performance comparison between the in-house ELISAs and a commercial test
To compare the performance of the in-house assays (Np & RBD ELISAs) against a commercially available test. The collected sera were additionally tested using a LFCI kit (The Diagnostic Kit for IgM / IgG Antibody to Coronavirus (SARS-CoV-2) (Lateral Flow), LIVZON, China). This LFCI test was widely used by the Chilean Health Services in the early stages of the pandemic, and it can separately detect IgG and IgM immunoglobulin isotypes, using Np as antigen. For this assessment, the LFCI interpretation criteria considered that any serum showing a signal for IgG or IgM was positive to SARS-CoV-2 infection. Statistical comparison between the LFCI and in-house assays are described below. To complement the comparison between the in-house ELISAs and the commercial LFCI test performance, a cost per sample analyzed at laboratory level was included. The latter evaluation only considered the cost of the reagents (in-house ELISAs) or the commercial value of the kit (LFCI).
Statistical analysis
Collected data were analyzed using GraphPad Prism 6 statistical software (USA) and the R software v4.0.2 [29]. They included mean values and standard deviation (mean ± SD) of the optical density (OD) distribution. Those values were subjected to a background subtraction step before data analysis. For comparative analysis of the infected and pre-pandemic OD distributions, a Mann-Whitney U-test was used. A receiver operating characteristic (ROC) analysis was conducted to determine the best OD cut-off for the in-house ELISA tests, where sample size was calculated using the equation proposed by Obuchowski, N [30], estimating a minimum of 43 infected and 43 uninfected to yield a study power of 80%, a significance level of 5%, and an expected area under the curve (AUC) of 65%. The ROC and AUC values, in addition to the sensitivity and specificity values, for Np or RBD antigens, were calculated by comparing infected (sera from confirmed COVID-19 patients) and non-infected specimens (sera from pre-pandemic individuals), based on a logistic regression model. The agreement between Np ELISA and RBD ELISA results were assessed using the Cohen’ kappa index and the paired McNemar’s test. Statistical differences in sensitivity and specificity distributions between Np and RBD ELISAs were evaluated using the Kolmogorov–Smirnov test. The overlap area between Np and RBD sensitivity and specificity curves was estimated using the “overlapping” package [31] for the R software. Finally, in-house ELISAs and LFCI results were cross tabulated in two-by-two tables and the agreement between them was also evaluated using Cohen’ kappa index and the paired McNemar’s test, in addition to the Kolmogorov–Smirnov test to assess statistical differences between the performance distributions of both tests, where Np ELISA and RBD ELISA were separately compared against the commercial LFCI test.
RESULTS
In-house ELISAs for specific SARS-CoV-2 antibodies using Np or RBD antigens
The Figure 1 shows a representative scheme of used proteins and their location in the SARS-CoV-2 virus, together with the OD distributions of the Np ELISA (left) and RBD ELISA (right) for confirmed cases and controls. At the comparison between duplicated runs, both assays showed an adequate consistency, and a low error level, with intra-assay variation of 11.2% and 13.4% for Np and RBD ELISAs, respectively. In this way, antibodies against Np protein showed that sera from COVID-19 patients had a significantly higher (97%) absorbance than sera from pre-pandemic individuals (1.02+/-0.12 vs 0.51+/-0,13 OD, respectively) (P < 0.0001). On the other hand, although to a lesser extent, the immunoassay against the RBD protein also showed that sera from patients with COVID-19 presented a significantly higher absorbance (43.5%) than sera from pre-pandemic individuals (1.09 +/-0.03 vs 0.71 +/-0.18 OD, respectively) (P < 0.0001). Interestingly, sera from confirmed COVID-19 patients showed a more homogeneous distributions in their absorbances with the RBD immunoassay than with Np (Figure 1, right and left, respectively). Furthermore, these distributions suggest that most of the analyzed sera presented antibodies that recognize and bound to both proteins (Np and RBD), and only 2 patients presented antibodies just against the RBD protein.
In-house ELISAs performance assessment (sensitivity and specificity estimation)
The ROC analysis of the Np ELISA indicated a mean sensitivity of 94% (95% CI = 83.5 – 98.8%) and a mean specificity of 100% (95% CI = 92.0 – 100%), with AUC of 0.99 (95% CI: 0.97 – 1.00) and an OD 450 nm positive cut-off value of 0.88 (Figure 2A). On the other hand, the ROC analysis of the RBD ELISA had a mean sensitivity of 96% (95% CI = 86.3 – 99.5%) and a mean specificity of 90% (95% CI = 78.3 – 97.5%), with AUC of 0.96 (95% CI: 0.91-1.00) and an OD 450 nm positive cut off value of 0.996 (Figure 3B). Those cut-offs represent the point in the ROC curves where the sensitivity and specificity were maximized for this set of samples.
The agreement between Np and RBD ELISAs was assessed as substantial to perfect (kappa=0.81, 95% CI = 0.69 – 0.93) based on the qualitative scale interpretation proposed by Landis and Koch [32] of Cohen’s kappa values. Additionally, the dichotomous ELISA results (positive/negative) were not statistically different (P = 0.18) between the two antigens. Although RBD ELISA presented a relatively higher sensitivity than Np ELISA, those differences were not statistically significant (P = 0.37), with an overlapping area between both curves of 69.7% (Figure 3A). Conversely, the use of RBD protein as antigen was associated to a statistically lower (P < 0.01) specificity, with a shared distribution of 10.8% between the Np and RBD proteins (Figure 3B).
In-house ELISAs and commercial LFCI test comparison
The cross-tabulation between the LFCI test results and SARS-CoV-2 infection status are presented in Table 1, whereas the cross-tabulation of the dichotomous ELISA and commercial test results are shown in Table 2. Based on the infection status (gold standard), the LFCI test presented a mean sensitivity of 94% (95% CI = 87.4 – 100%) and a mean specificity of 100% (95% CI = 100 – 100%). The Cohen’ kappa index showed a substantial to perfect (kappa=0.89, 95% CI = 0.80 – 0.98) agreement between the Np ELISA and the LFCI test, whereas the agreement between RBD ELISA and the LFCI test was categorized as substantial kappa=0.78, 95% CI = 0.66 – 0.91). Non-significant differences in the sensitivity (P = 0.999) or specificity (P = 0.999) distributions were observed for the comparison between Np ELISA and the LFCI test. Conversely, the comparison between RBD ELISA and the LFCI test render a significant difference in the specificity distribution (P < 0.01), but non-differences were observed between them for the sensitivity distributions (P = 0.999). The use of the LFCI had a cost of $10.57 USD per serum analyzed, whereas the Np ELISA and RBD ELISA had costs of $4.23 USD and $2.65 USD per serum, respectively. Thus, the in-house ELISAs were between 2.5 – 4.0 times cheaper than the used commercial test.
DISCUSSION
The present research reports the performance estimation of two in-house ELISA assays for the surveillance of SARS-CoV-2 infection, which showed a comparable performance than a commercial test but at a fraction of the cost. Thus, they may be alternative tools for conducting large epidemiological studies in resource limited environments, such as developing countries. However, the in-house ELISAs require more qualified personnel than the commercial kit, in addition to be more time consuming. In the present study, labor was not considered in the cost comparison. Despite the latter, it is important to note that full-time technicians are commonly present at university laboratories, where a more frequent problem, in developing countries, is the lack of resources to buy reagents rather than labor availability. The OD cut-off optimization and performance estimates considered local epidemiological conditions; thus, study results represent the expected performance of the assays in the target population. ROC analysis indicated a non-significant higher sensitivity and statistically significant lower specificity of immunoglobins targeting RBD in comparison to Np. In opposition, previous studies have shown a higher Np sensitivity than RBD, and higher specificity of RBD than Np [22,23]. Even though these authors argue that this phenomenon occurs in the early phase of infection, with IgG as the main isotype. In this line, Burbelo et al. [17] showed in a longitudinal study that antibodies against Np emerged before antibodies against the S protein. In the present study, confirmed COVID-19 cases had manifested clinical signs before serum collection (median = 11.5 days, IQR: 9 – 15 days). Thus, sensitivity and specificity results reflect the performance of Np and RBD antigens at later stages of the infection under local conditions. Moreover, considering that the intended use of the in-house ELISA assays is for surveillance purposes, study outputs could represent the expected performance in the general population, for the detection of past infections of SARS-CoV-2. A recent meta-analysis study [19], assessing the performance of serological tests for the detection of antibodies against SARS-CoV-2, has shown that ELISA tests (detecting IgG or IgM) present lower sensitivity than the assays evaluated in this study, where on average those assays presented a sensitivity of 84.3% (95% CI: 75.6 – 90.9%). On the other hand, that meta-analysis showed a comparable specificity (range 96.6 – 99.7%), at least for Np ELISA. In this sense, the combined capture of three immunoglobulins, and particularly the inclusion of IgA, could contribute to explaining the higher sensitivity observed. Similarly, Ma et al. [28] reported that measuring IgA in addition to IgG or IgM would increase the sensitivity of the assay. In this line, previous reports have indicated that SARS-CoV-2 elicits robust humoral immune responses, including the production of IgM, IgG, and IgA immunoglobulin isotypes [33]. Patients have been shown to achieve seroconversion and produce detectable antibodies within 20 days from the symptom onset, though the kinetics of IgM and IgG production was variable [4,34,35]. A serological screening that includes the detection of IgA, IgM and IgG could be more consistent as a strategy to prevent the spread of the virus, given the need to maximize the sensitivity [13], when the diagnostic assay is used as a screening test.
Based on the results of the Np and RBD in-house ELISAs, the combined detection of three immunoglobulin isotypes (IgA, IgM and IgG) could be more sensitive than assays that only detect one or two immunoglobulin isotypes, particularly for screening purposes or in studies to evaluate the exposition levels in the general population to SARS-CoV-2 virus, regardless of stage, severity, and symptoms of COVID-19 disease. However, the increase in sensitivity is offset by the reduction in specificity, which was especially evident when the RBD antigen was used. This could be also appreciated when contrasting OD distributions for both antigens (Figure 2), where the comparison between control sera showed that RBD ELISA tend to have higher OD values than Np ELISA in this group, indicating a lower specificity of this antigen.
CONCLUSIONS
Considering the intended purpose of the in-house ELISA, a relevant element was to obtain accurate sensitivity and specificity estimates, to support further epidemiological studies. In this sense, estimates for Np ELISA presented a variation (in relation to the mean) between 8 – 16%, whereas RBD ELISA presented a variation between 14 – 21%. Thus, relatively precise estimates were obtained. Moreover, at the comparison with the commercial LFCI test, the Np ELISA presented a comparable performance than the LFCI test, while the RBD ELISA presented a significant lower specificity than the LFCI test and the Np ELISA. Therefore, it can be concluded that the Np ELISA is a better assay than RBD ELISA, which presented performance comparable to a commonly used commercial test, but at a quarter of the cost, making it a viable alternative for surveillance studies in developing countries.
Data Availability
Data cannot be shared publicly because of Chilean law. Data disclosure should be requested to Scientific Ethics Committee of the Servicio de Salud Valdivia (SSV), Ministry of Health of Chile
DECLARATIONS
Competing interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Author Contributions
CV (Cristobal) and ARR conceived the idea; CV (Cristobal), ARR, AF, MN, FVE, NC, and AP designed sampling collections and experimental procedures. GV, CA, GN performed laboratory analysis. CV (Cristobal), GA, and CV (Claudio) made the statistical analysis. All the authors wrote, discussed, and approved the final version of this manuscript.
Funding
This work received financial support from the National Agency for Research and Development (ANID) of the Ministry of Sciences of Chile, grant number: ANID COVID N°0585.
Acknowledgements
A special thanks to Dr. Ratto (Animal Science Institute, UACh) for the facilities granted for access and the use of the photo-documentation machine during the COViD-19 pandemic and to the Graduate School of the Faculty of Veterinary Sciences (UACh) for co-financing the acquisition of the plate reader (HR801, Shenzhen Highcreation Technology Co.,Ltd). We also thank the SEREMI Coquimbo region, Servicio de Salud de Coquimbo, Hospital Regional de Ovalle, and Municipality of Monte Patria for their logistical support, and Vinka Valencia, Paula Rojas, Cristian Brevis and Paxelia Huertas for their invaluable assistance during field work for the pre-pandemic sampling. Pre-pandemic sampling was supported by EU-LAC Health (EULAC/FONIS T020067), through the PERITAS (Molecular epidemiological studies on pathways of transmission and long-lasting capacity building to prevent cystic echinococcosis infection) project (G.A. EULACH16/T02-0067; https://www.iss.it/en/web/iss-en/who-cc-peritas).