Evaluation of six commercial SARS-CoV-2 Enzyme-Linked Immunosorbent assays for clinical testing and serosurveillance ==================================================================================================================== * Suellen Nicholson * Theo Karapanagiotidis * Arseniy Khvorov * Celia Douros * Francesca Mordant * Katherine Bond * Julian Druce * Deborah A Williamson * Damian Purcell * Sharon R Lewin * Sheena Sullivan * Kanta Subbarao * Mike Catton ## ABSTRACT **Background** Serological testing for SARS-CoV-2 complements nucleic acid tests for patient diagnosis and enables monitoring of population susceptibility to inform the COVID-19 pandemic response. As we move into the era of vaccines, the detection of neutralising antibody will become increasingly important. Many serological tests have been developed under emergency use authorization, but their reliability remains unclear. **Methods** We evaluated the performance of six commercially-available Enzyme-linked Immunosorbent Assays (ELISAs), including a surrogate virus neutralization test, for detection of SARS-CoV-2 immunoglobulins (IgA, IgM, IgG), total or neutralising antibodies and a subset of results were compared to microneutralisation. **Results** For sera collected > 14 days post-symptom onset the Wantai total Ab performed best with highest sensitivity 100% (95% confidence interval: 94.6-100) followed by 93.1% for Euroimmun NCP-IgG,93.1% for GenScript Surrogate Virus Neutralization Test, 90.3% for Euroimmun S1-IgG, 88.9% for Euroimmun S1-IgA and 83.3% for Wantai IgM. Specificity for the best performing assay was 99.5% and for the lowest 97.1%. **Conclusion** Wantai ELISA, detecting total immunoglobulins against SARS-CoV-2 receptor binding domain, had the best performance. Antibody target, timing and longevity of the immune response, and the objectives of testing should be considered in test choice. ELISAs should be used within a confirmatory testing algorithm to ensure reliable results. ELISAs provide high quality results, with flexibility for test numbers without the need for manufacturer specific analyzers. Key words * SARS-CoV-2 * COVID-19 * serology * ELISA * antibody * neutralising ## INTRODUCTION Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) the causative agent of Coronavirus disease 2019 (COVID-19) has infected > 71 million people and caused > 1.6 million deaths globally as of 18 December 2020 [1]. COVID-19 manifests as an acute respiratory illness, although asymptomatic infections occur [2]. Laboratory diagnosis of COVID-19 primarily relies on detection of viral RNA by RT-PCR in respiratory tract samples or antigen tests, the sensitivity of which declines as infection resolves. In contrast, antibody (Ab) is detected in most individuals 10-15 days following onset of symptoms [3]. Serological testing can detect previous infection in people who have recovered without RT-PCR testing, who are RT-PCR negative [4, 5], or whose RT-PCR results are difficult to interpret [6]. Serology may also inform our understanding of antibody longevity and quantification of neutralising antibody (nAb) response in vaccine trials [6, 7, 8]. As the pandemic progresses estimates of prior exposure and population prevalence from serological assays will play an increasingly important role for public health decision making [9, 10]. However, there is substantial variation in assay performance and correlation to neutralising antibodies [11]. In this study we evaluate the sensitivity and specificity of six commercially available serological ELISA tests using either spike (S), nucleocapsid (NCP) or receptor binding domain (RBD) antigens for detection of specific isotypes of SARS-CoV-2 (IgA, IgM, IgG), total Ab or nAb (using a surrogate virus neutralization test (sVNT)). We also compared a subset of ELISA results to microneutralisation (MN), the current gold standard assay for SARS-CoV-2-specific antibody. ## METHODS ### Study Design A retrospective study evaluating the sensitivity and specificity of commercially available ELISAs for detection of Abs to SARS-CoV-2 virus on well pedigreed sera. ### Evaluation serum samples Sera from three patient groups were assembled to assess aspects of assay performance: (1) Infected group: sera from patients with prior RT-PCR confirmed SARS-CoV-2 infection for the assessment of sensitivity. Specificity was calculated for (2) Population group: pre-pandemic sera representing the Victorian population, collected between 2011 and 2018 and (3) Cross-reactive group: pre-pandemic sera for assessment of potential cross-reactivity; from patients with seasonal coronavirus, SARS-CoV1, MERS-CoV or other non-COVID-19 acute infections (see Table 1 and Table 3 for the full list). View this table: [Table 1.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/T1) Table 1. Description of Evaluation Samples for the assays Samples were obtained from the Victorian Infectious Diseases Reference Laboratory (VIDRL) at The Peter Doherty Institute for Infection and Immunity and Royal Melbourne Hospital (RMH), Victoria, Australia (Table 1). ### Testing protocols #### RT-PCR SARS-CoV-2 RNA was detected in respiratory swabs by RT-PCR and results provided by both institutions using either in-house RT-PCR with previously published primers [12] used by VIDRL, or a commercial Coronavirus Typing assay, used by RMH [13]. All positive samples were confirmed by a second assay/target. #### ELISA ELISA testing was performed in the Serology Laboratory at VIDRL following manufacturers’ Instructions for Use (IFU) with results reported semi-quantitatively as either a signal/cut-off ratio (Euroimmun and Wantai) or percentage inhibition (sVNT). After initial sVNT testing we also repeated testing if results were within 10% of the IFU-suggested cut-off (18%-22%). Assay performance characteristics were assessed using the panels described previously. Testing was performed manually though high throughput testing could be performed using robotic platforms. Intra-assay variability was calculated by testing 10 in-house quality control sample replicates within the same microtitre plate and inter-assay variability by testing on different plates on different days using the same kit lot number; results are reported as a coefficient of variation (CV). ### Euroimmun anti-SARS-CoV-2 S1-IgG, S1-IgA and anti-SARS-CoV-2 NCP-IgG (Euroimmun Medizinische Labordiagnostika, Lubeck, Germany) These are indirect ELISAs for detection of immunoglobulin (Ig) class IgG or IgA against SARS-CoV2 antigens. Wells are coated with either (i) S protein S1-domain or (ii) modified NCP. SARS-CoV-2 binding antibodies are detected using enzyme-labelled anti-human-IgG or anti-human-IgA conjugates and a colourmetric substrate and are read spectrophotometrically. ### Wantai SARS-CoV-2 total Ab (Beijing Wantai Biological Pharmacy Enterprise, Beijing China) This is a two-step incubation antigen ‘sandwich’ assay detecting total antibodies binding the SARSCoV-2 RBD within the S1-subunit of S protein. Patient antibody to SARS-CoV-2 that binds antigen coated on the plate, is bound by horseradish peroxidase (HRP) antigen conjugate forming an antigen-antibody-antigen-HRP complex detectable by colourmetric substrate and read spectrophotometrically. ### Wantai SARS-CoV-2 IgM This is a capture ELISA for detection of IgM-class antibodies to SARS-CoV-2 virus. Anti-µ chain antibodies on the plate capture patient IgM antibodies; detection is by recombinant SARS-CoV-2antigen-HRP-conjugate followed by a colourmetric substrate read spectrophotometrically. ### GenScript SARS-CoV-2 Surrogate Virus Neutralization Test (sVNT) (GenScript USA, Inc. New Jersey, USA) This is a species and isotype independent blocking ELISA which mimics virus neutralisation, detecting circulating neutralising SARS-CoV-2 antibodies that block the interaction between the viral spike RBD and host angiotensin converting enzyme 2 (ACE2) cell surface receptor. HRP-conjugated recombinant SARS-CoV-2 RBD fragment binds to any circulating nAb to RBD preventing capture by the hACE2 protein on the well which is subsequently removed in the following wash step. Substrate reaction incubation time is determined by temperature; ideal reaction temperature and time in the IFU are 25°C for 15 minutes. For temperatures lower than 25°C, the time can be extended. At 15 minutes our control values did not meet the assay validity criteria, but at 20 minutes they fell within the acceptable ranges. Colour intensity is inversely dependent on the titre of anti-SARS-CoV-2 nAbs. We report three types of alternative estimates: (i) 20% cut-off without retesting (as per the IFU), (ii) 20% cut-off with retesting for equivocal results (18-22%) and (iii) 25% cut-off without retesting. ### Microneutralisation assay In a subset of sera, a comparison was performed using an in-house MN assay and the commercial ELISAs. To compare MN sensitivity and specificity to the Euroimmun and sVNT assays, ninety samples from RT-PCR positive patients were assessed (Panel A). Seventy samples were assessed for comparison of MN to the Wantai assays, which require a higher volume of sera than the other assays (Panel B). The MN assay was performed using an in-house assay as described previously [13]. Briefly, the ability of serial 2-fold dilutions of sera to neutralize the infectivity of 100 median tissue culture infectious doses of SARS-CoV-2 was assessed by inhibition of viral cytopathic effect in Vero cells. The nAb titre was calculated using the Reed/Muench method [14, 15]. ### Statistical analysis All statistical analyses were conducted using R version 4.0.2 [16]. Responses were assessed in each of the 3 patient groups (Infected, population, cross-reactive). Sensitivity was estimated using the infected group separately for each of three categories from time of symptom onset: (<7 days, 7-14 days, >14 days). Each observation was treated as independent within each subgroup. Specificity was estimated separately for the population group and the cross-reactive group. The 95% confidence intervals were generated using the exact binomial Clopper-Pearson method with PropCIs R package [17]. Sensitivity estimates (point estimates and CI bounds) were averaged across the onset categories to give the “averaged” sensitivity estimate. Point estimates and interval bounds for the sensitivity and the population group specificity were used to calculate the corresponding estimates and bounds for PPV and NPV at different theoretical levels of population prevalence (0.1%, 0.5%, 1%, 5%, 10% and 20%). ### Ethics Project ethical approval for RMH specimens was obtained from Melbourne Health Human Research Ethics Committee (RMH HREC QA2020052). The in-house panel consisted of anonymised excess diagnostic specimens sent to VIDRL for COVID-19 testing and the ‘VIDRL Serum Reference Collection’. ## RESULTS ### Sensitivities and specificities of six ELISA assays Performance characteristics of the six commercial assays are shown in (Table 2). Increased sensitivity as well as increased antibody level, reported as index values or % inhibition, was observed across all timeframes for all assays (Figure 2). For comparative assay sensitivity and specificity see (Figure 1). View this table: [Table 2.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/T2) Table 2. Performance Characteristics of assays with RT-PCR, all time points and average. View this table: [Table 3.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/T3) Table 3. Cross-reactivity results for non-COVID-19 mixed infection pre-pandemic group ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/01/26/2021.01.21.21250249/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/F1) Figure 1. Comparative analysis of assay sensitivity post symptom onset < 7 days, 7-14 days, ≥ 14 days and averaged. Comparative analysis of assay specificity in cross-reactive assessment and population groups Abbreviations: E-S1-IgA, Euroimmun S1-IgA; E-S1-IgG, Euroimmun S1-IgG; E-NCP-IgG, Euroimmun NCP-IgG; sVNT, GenScript surrogate virus neutralization test 20% cut-off with repeat testing for equivocal results (18-22); sVNT-20 with 20% cut-off with no repeat testing; sVNT-25 with 25% inhibition cut-off and no repeat testing; W-IgM, Wantai IgM, W-T, Wantai Total Ab. ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/01/26/2021.01.21.21250249/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/F2) Figure 2. Boxplots of data distribution, as signal/cut-off value for each assay for the RT-PCR Positive and control sera by days post symptom onset; <7 days, 7-14 days and >14 days. Boxes represent median values and interquartile range, and whiskers represent largest and smallest values. Grey horizontal lines represent the cut-off value, and the shaded grey indicates the equivocal/borderline zone. Measure is index value (sample optical density (OD) value/ cut-off OD) for all assays except sVNT. sVNT measure is %inhibition, calculated as per IFU. #### Euroimmun S1 IgA, S1 IgG and NCP IgG Sensitivity was assessed using sera from subjects with RT-PCR proven COVID-19 disease (Table 2). When sera collected > 14 days post-symptom onset were considered, sensitivity was 88.9% (79.395.1) for S1-IgA, 90.3% (81.0-96.0) for S1-IgG and 93.1% (84.5-97.7) for NCP-IgG. The averaged sensitivity across all timeframes was 58.0% (44.4-71.1) for S1-IgA, 46.7% (35.4-59.6) for S1-IgG and 51.4% (39.4-64.2) for NCP IgG. Specificity was assessed using the population and cross-reactive groups (Table 2). Specificity when testing the pre-pandemic population group was lowest at 95.0% (90.7-97.7) using the S1-IgA, 97.2% (93.6-99.1) using S1-IgG and highest 99.0% (96.3-99.9) using NCP-IgG. In the cross-reactive assessment group, the lowest specificity was 83.3% (65.3-94.4) using NCP-IgG, 86.7% (69.3-96.2) using S1-IgA and highest using S1-IgG at 90% (73.5-97.9). Initial testing of the S1-IgA kit gave poorer specificity than reported here; however, with the introduction of a new buffer by Euroimmun, retrospective testing of pre-pandemic samples showed an 8.5% specificity improvement. Cross-reactivity was observed with sera containing anti-SARS-CoV-1 antibody (all tests), anti-MERS antibody (only NCP IgG, 1/9 samples), and sera positive for mycoplasma and CMV (all tests) and dengue (only S1-IgA, 1/5 samples) antibodies. No cross-reactivity was observed to sera positive for parvovirus-B19 and seasonal coronavirus antibodies (Table 3). #### GenScript SARS-CoV-2 sVNT For the COVID-19 RT-PCR positive group, sensitivity of the sVNT was 93.1% (84.5-97.7) when sera collected > 14 days from symptom onset were considered and was 54.4% (42.3-66.7) across all time periods (Table 2). The assay specificity was 97.1% (94.6-98.7) when the pre-pandemic population group were tested. All other assays required re-testing of samples with results in the ‘equivocal zone’, 10-20% of the assay cut-off, depending on the assay. Because the sVNT had similar intra-assay and inter-assay variability to the other assays, we wanted to see if specificity improved with retesting of samples in the 10% equivocal zone. When the pre-pandemic population group sera were tested with repeat testing of equivocal zones specificity, increased to 99.4% (97.7-99.9). With a 20% cut-off and equivocal zone repeat testing 2/312 samples gave a false positive result (both MN negative), both were close to the 20% inhibition cut-off (22.5%, 21%). Because the IFU suggests setting a population specific cut-off, we considered a range of thresholds (Supplementary Figure 1a, 1b, 1c & Table 1a, 1b). Using a cut-off value of 25% increased specificity and PPV without changing the sensitivity and NPV (Table 2). In the cross-reactivity assessment group cross-reactivity was observed with sera containing antibody to SARS-CoV-1 (Table 3). #### Wantai Total Ab and IgM The averaged sensitivity for the Wantai Total Ab was 71.6% (52.8-87.2) and for the IgM assay 61.6% (41.8-80.8) which improved to 100.0% (94.6-100.0) and 83.3% (72.1-91.4), respectively, when restricted to sera collected > 14 days post-symptom onset. (Table 2). The specificity for the Wantai Total Ab was 99.5% (97.4-100.0) and for IgM was 99.0% (94.6-100.0) in the population group and for the cross-reactive group these were 97.2% (85.8-99.9) and 96.7% (82.8-99.9), respectively. Cross-reactivity was observed with sera containing antibody to SARS-CoV1 (Table 3). ### Positive Predictive Value (PPV) and Negative Predictive value (NPV) PPV and NPV were calculated across a range of population prevalence estimates (Figure 3). In a low prevalence setting like ours, assuming population prevalence of 0.32% for Victoria and 0.11% for Australia [18] PPV for the best performing assay, Wantai Total Ab was 40.2% and 18.7%, respectively while the NPV was 100% for sera collected >14 days post-symptom onset (Supplementary Table 2). ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2021/01/26/2021.01.21.21250249/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/F3) Figure 3. Performance characteristics of the assays across a range of population prevalence estimates: 0.1%, 0.5%, 1%, 5%, 10% and 20%. Abbreviations: E-S1-IgA, Euroimmun S1-IgA; E-S1-IgG, Euroimmun S1-IgG; E-NCP-IgG, Euroimmun NCP-IgG; sVNT, GenScript surrogate virus neutralization test 20% cut-off with repeat testing for equivocal results (18-22); sVNT-20 with 20% cut-off with no repeat testing; sVNT-25 with 25% inhibition cut-off and no repeat testing; W-IgM, Wantai IgM, W-T, Wantai Total Ab. ### Discordance between assays In the RT-PCR positive group there were 41 discordant results, with discordance increasing with time since disease onset from 8/41 at < 7 days post-onset to 19/41 at > 14 days post-onset. In the population and cross-reactive assessment groups there were 17 and 6 discordant results, respectively. (Supplementary Figure 2– Heat-map). ### Intra and Inter-assay variability Intra-assay variability was below 10%, ranging from CV=3.1% for the Euroimmun S1-IgA to CV=8.8% for the Euroimmun S1-IgG. Inter-assay variability ranged from Euroimmun NCP-IgG CV=3.1% to Wantai IgM CV=14.9% (Supplementary Table 3a, 3b). ### Assay comparison with microneutralization Eighty-five samples from RT-PCR positive Panel A were tested by MN (Table 4a). Seventy three percent (62/85) were MN positive. Euroimmun S1-IgA had equal sensitivity with 73% (62/85) samples positive, 67% (57/85) sVNT positive, 67% (56/84) Euroimmun NCP IgG positive and 62% (53/85) Euroimmun S1 IgG positive. Hence MN and Euroimmun S1-IgA had the highest sensitivity. At > 14 days post symptom onset Euroimmun S1 IgA and Euroimmun NCP-IgG had 100% (38/38) sensitivity, followed by MN and Euroimmun S1 IgG at 97% (37/38). sVNT was 95% (36/38) sensitive. View this table: [Table 4a.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/T4) Table 4a. Microneutralisation comparison with ELISAs View this table: [Table 4b.](http://medrxiv.org/content/early/2021/01/26/2021.01.21.21250249/T5) Table 4b. Microneutralisation comparison with ELISAs (% agreement) Seventy samples from RT-PCR positive Panel B were tested by MN. Seventy nine percent (55/70) were MN positive. Wantai Total had highest sensitivity with 87% (61/70) positive. At > 14 days post symptom onset Wantai Total Ab and MN had comparable sensitivities of 100% (41/41) and 98% (40/41) respectively, and Wantai IgM was 88% (36/41) sensitive (Table 4a). The MN assay specificity for the population and cross-reactive assessment sera groups was 100%. ## DISCUSSION Reliable COVID-19 serosurveillance is important for guiding the pandemic response [9, 10]. At the individual level, serology can provide a tool for resolving the diagnosis for patients with infections not confirmed by RT-PCR. At the population level, it provides policy-makers with an assessment of overall impact of the pandemic and vaccination efficacy. Assay sensitivities ranged from 46.7% to 71.6%; and specificities from 95.0% to 99.5% (pre-pandemic population group), and from 83.3% to 97.2% (cross-reactivity assessment group). Sensitivity for sera collected >14 days post-symptom onset ranged from 83.3% to 100%. Consistent with the biology of an immune response, and as reported by others, when results from all time-points were considered, sensitivity was low, but increased substantially when only sera collected >14 days post-onset was assessed [19], underlining the need to wait a sufficiently long period before confirming infection status by serology. The Euroimmun NCP-IgG and sVNT had good sensitivity, however the highest sensitivity was achieved by Wantai Total Ab at 100%. The Euroimmun NCP-IgG and Wantai IgM had better pre-pandemic population group specificity at 99%, than sVNT. Adjusting the sVNT cut-off to 25% improved its specificity to 100% (Table 2). Wantai Total Ab achieved highest specificity in the cross-reactivity assessment group at 97.2%. The intra and inter-assay variability of sVNT was not dissimilar to the other assays. All other assays employ an ‘equivocal’ zone, between 10-20% below and above the cut-off. We observed improved sVNT specificity when we varied from the IFU to use a +/-10% repeat equivocal zone. Cross-reactivity was observed in all assays with sera containing anti-SARS-CoV-1 antibody. Cross reactivity was more commonly seen in the Euroimmun assays, and depending on the assay, in sera positive for MERS, mycoplasma, CMV and dengue antibodies, consistent with other reports [19]. Assays challenged with seasonal coronavirus antibody positive sera were not reactive, and no cross reactivity to sera containing parvovirus B-19 antibodies was seen (Table 3) Test result discrepancies were observed in the SARS-CoV-2 RT-PCR positive group. In part this appeared attributable to the type of antibody being detected, the antigen targeted in the assay and the assay format. For samples collected > 14 days post-onset, 11 were Wantai IgM negative but Total Ab positive. All 11 samples were collected 21 to > 30 days post onset, suggesting discordance attributable to IgM loss [20]. We retrospectively tested these samples for IgA (results not shown). Six of eleven (55%) were IgA positive, suggesting waning IgM antibody and to a lesser extent IgA. The Wantai Total Ab and sVNT (using a modified population specific cut-off of 25%) demonstrated the highest specificity. This may be because both assays detect Ab to the RBD within the S1-subunit, therefore increasing specificity by exclusion of cross-reacting epitopes outside this domain [19]. sVNT is the only immunoassay correlating with nAb [21,22] although studies have shown samples with Wantai Total Ab positive results with an index ratio > 10 had detectable levels of nAb by VNT [11]. Assay formats which utilise a single detector antibody, for example Wantai Total Ab, may show greater specificity than those using two antibodies in an antigen-antibody-antibody format, such as the Euroimmun assays [23]. With the exception of the Wantai Total Ab, manufacturers for all assays reported higher sensitivities than we demonstrated. However, direct comparison of results is limited by differences in available sample cohorts and sampling timeframes. Our results are consistent with prior reports of sensitivity for Wantai Total Ab of 100% for samples collected > 14 days post-onset [11]; Euroimmun S1-IgG sensitivity post-onset, 43% (7 to 13 days), 67% (14 to 20 days) and 78% (≥ 21 days) and specificity. 96.0% [19]; Euroimmun NCP-IgG 88.89% (>14 days) [22]; and 95.2% (14-17 days), and specificity 94.7% [24]. While other groups have reported slightly higher sVNT sensitivity (95-100%) and specificity 100% [22] than we observed, sensitivity in our assessment was 93.1% (95% CI 84.5-97.7) for sera collected > 14 days post-symptom onset, with specificity 97.1% (pre-pandemic population group). This improved to 100% with use of a modified 25% inhibition cut-off value: and 96.7% for the cross-reactive group. We compared agreement between the ELISAs and an in-house MN assay, which is the current gold standard to assess protective immunity against SARS-CoV-2 [11]. The Wantai IgM assay had the highest agreement with MN at 91%, followed by Wantai Total Ab at 86% and the lowest was with the Euroimmun S1-IgG at 79%. The sVNT performed according to the IFU gave the best concordance with MN at 84% compared to 83% and 81% for the alternate methods previously described. At > 14 days post symptom onset Wantai Total Ab had the best agreement with MN at 98% followed by Euroimmun S1-IgA, Euro NCP-IgG and sVNT (as per IFU) with (97%, 97% and 95% respectively) and Wantai IgM having the poorest agreement at 93%. MN and ELISA result concordance for sera from the pre-pandemic population group ranged from 55-100% and from 0-80% for the cross-reactive assessment group; MN was the only assay not to show reactivity to the SARS-CoV-1 antibody positive sample. The sVNT was not as sensitive as MN in our comparison. However, it is simpler to perform than a VNT, does not require a level 3 Biocontainment laboratory, a highly skilled operator nor 5-day test turnaround time, and it has high specificity (if a population adjusted cut-off is employed), and generally good correlation with VNT [25]. Further comparison is required to determine its suitability as an alternative to VNT, in certain settings, particularly as we move into the post-vaccination phase. In our study, as reported by others, IgA and IgM were detected earlier than IgG, and IgA specificity was lower than IgM or IgG [20]. For clinical testing, the Wantai total antibody assay demonstrated the best overall performance with highest sensitivity, specificity, PPV and NPV. Detecting RBD-Ab, the strong linear correlation between S- and RBD-Ab and ACE-2 receptor binding, suggests nAb is being detected [11, 25]. It is also suitable for population screening where high specificity is required, and high sensitivity is desirable. Other assays such as Euroimmun NCP-IgG have high sensitivity but lower specificity, giving it a low PPV. Further research on the utility of IgA and IgM detection is required, especially as indicators of recent infection. The high specificity sVNT assay was further improved to 100% with cut-off adjustment to 25% for our population. Although sensitivity of the sVNT was lower than the Wantai Total Ab, its specificity makes it a reliable supplemental test to use with screening assays. The assays evaluated here detect antibodies to different viral targets: S, NCP or RBD and emerging humoral response dynamics studies suggest these antibodies have different half-lives [26], an important consideration for serosurveillance. An additional important consideration for assay selection is understanding whether the assay detects non-nAb or nAb (particularly for testing post-vaccination); however further research is required. The Wantai SARS-CoV-2 total Ab assay was alone among those tested in our hands in achieving sensitivity and specificity very close to that reported by the manufacturer (94.5%; and 100% respectively); one limitation of this assay is the requirement for 100ul of serum versus 10ul for the other assays. In low prevalence populations and with low test PPV’s, serological testing algorithms should use highly sensitive assays for screening such as Wantai Total Ab, followed by supplemental or confirmatory testing with highly specific assays such as sVNT or MN to ensure reliable results. An important component of this study is the comparison of sensitivities of the ELISAs and MN, which should be considered in the context of discordant screening and supplemental results in testing algorithms. Limitations of our study include the smaller number of convalescent patient RT-PCR-POS confirmed SARS-CoV-2 samples collected > 14 days post-symptom onset used for the Euroimmun and sVNT evaluations; Panel A, 49% (72/147) of samples compared to Panel B, 69% (66/96) of samples used for Wantai Total Ab and IgM assays. A rise in index value/inhibition for all assays was observed over time (Figure 2) however these measurements are semi-quantitative, indicating antibody amount present for comparison between assays, not antibody titre which can only be obtained by sample titration to ‘end-point.’ In conclusion, our study revealed different performance characteristics between six commercially available ELISAs with Wantai Total Ab demonstrating best overall performance. As we move into the post-vaccination phase consideration should be given to the use of GenScript sVNT in certain settings, as an alternative to VNT. While a positive ELISA result with neutralising activity, even if specific, provides evidence of prior SARS-CoV-2 infection, more research is required to determine correlates of protection, antibody longevity and the role played by the adaptive immune system. ## Supporting information Supplementary Data [[supplements/250249_file02.pdf]](pending:yes) ## Data Availability All data regarding the study is available on request. ## Conflicts of interest The authors declare no competing interests. ## ACKNOWLEDGEMENTS The authors thank the Serology Laboratory at VIDRL, particularly Di Karamalakis, The Viral Identification Laboratory and the Information Technology department, particularly Dallas Wilson. We thank Dr. Allen Cheng, Dr. Maryza Graham and Megan Wieringa from Monash Medical Centre and Dr. Katherine Kedzierska from The Department of Microbiology and Immunology, The University of Melbourne at The Doherty Institute, for referring samples to VIDRL for COVID-19 investigation. Annette Fox from WHO Collaborating Centre for Reference and Research on Influenza, VIDRL, Doherty Institute for her contribution to the study. Prof. Lin Fa Wang for supplying us with the sVNT assay for evaluation. The Victorian Infectious Diseases Reference Laboratory is supported by the Victorian Department of Health and Human Services. The WHO Collaborating Centre for Reference and Research on Influenza is supported by the Australian Government Department of Health. * Received January 21, 2021. * Revision received January 21, 2021. * Accepted January 26, 2021. * © 2021, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.Control ECfDPa. Cohen A, Kessel B. False positives in reverse. COVID-19 situation up-date worldwide, as of 27 November 2020.Available at: [https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases](https://www.ecdc.europa.eu/en/geographical-distribution-2019-ncov-cases). 2. 2.Gudbjartsson D, Norddahl G, Melsted K et al. Humoral Immune Response to SARS-CoV-2 in Iceland.1 Sept 2020, NEJM. Available at: [https://doi:10.1056/NEJMoa2026116](https://doi.org/10.1056/NEJMoa2026116) 3. 3.Seow J, Graham C, Merrick B et al. Longitudinal evaluation and decline of antibody responses in SARS- CoV-2 infection. medRxiv 20148429 [Preprint]. 11 July 2020 [cited 11 Oct 2020]. Available from: [https://doi.org/10.1101/2020.07.09.20148429](https://doi.org/10.1101/2020.07.09.20148429). 4. 4.Zhang W, Du R-H, Li B, et al. Molecular and serological investigation of 2019-nCoV infected patients: implication of multiple shedding routes. Emerg. Microbes Infect 2020; 9: 386–89 5. 5.SARS-CoV-2 use cases white paper, Mickey Urdea, Halters Associates. Rockefeller Foundation, Arizona State University and Duke University, USA. [www.halteresassociates.com](http://www.halteresassociates.com). 6. 6.Yong S, Anderson D, Wycliffe W et al. Connecting cluster of COVID-19: an epidemiological and serological investigation. Lancet Infect Dis 2020; 20: 809–15. Available at: [https://doi.org/10.1016/S1473-3099(20)30273-5](https://doi.org/10.1016/S1473-3099(20)30273-5) [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/s1473-3099(20)30273-5&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32330439&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F26%2F2021.01.21.21250249.atom) 7. 7.Qiu H, Wu J, Hong L, et al. Clinical and epidemiological features of 36 children with coronavirus disease 2019 (COVID-19) in Zhejiang, China: an observational cohort study. Lancet Infect Dis 2020;20 (6), p689–696, June 01, 2020. 8. 8.Public Health England, Porton Down, Nuffield Department of Medicine, University of Oxford, Oxford University Hospitals NHS Foundation Trust. Evaluation of sensitivity and specificity of four commercially available SARS-CoV-2 antibody immunoassays, July 2020. 9. 9.Halters Associates. Use cases for SARS-CoV-2 assays 2020. Available at: [https://haltersassociates.com/halteres-sars-cov-2-use-case-tables/](https://haltersassociates.com/halteres-sars-cov-2-use-case-tables/). 10. 10.Lerner A, Eisinger R, Lowy D et al. The COVID-19 serology studies workshop: recommendations and challenges. Immunity 2020; 53:1–5. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.immuni.2020.06.012&link_type=DOI) 11. 11.Corman VM, Landt O, Kaiser M, Molenkamp R, Meijer A, Chu DK, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro surveillance: bulletin Europeen sur les maladies transmissible = European communicable disease bulletin. 2020; 25(3). 12. 12.Bond K, Nicholson S, Lim S et al. Evaluation of Serological Tests for SARS-CoV-2: Implications for Serology Testing in a Low-Prevalence Setting. J Infect Dis 2020.222(8):1280–1288. PMID: 32761124. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/infdis/jiaa467&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32761124&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F26%2F2021.01.21.21250249.atom) 13. 13.Houser KV, Gretebeck L, Ying T, et al. Prophylaxis with a Middle East respiratory syndrome coronavirus (MERS-CoV)- specific human monoclonal antibody protects rabbits from MERS-CoV infection. J Infect Dis 2016; 213:1557–61. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/infdis/jiw080&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26941283&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F26%2F2021.01.21.21250249.atom) 14. 14.Subbarao K, McAuliffe J, Vogel, et al. Prior infection and passive transfer of neutralizing antibody prevent replication of sever acute respiratory syndrome coronavirus in the respiratory tract of mice. J Virol 2004; 78:3572–7. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoianZpIjtzOjU6InJlc2lkIjtzOjk6Ijc4LzcvMzU3MiI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIxLzAxLzI2LzIwMjEuMDEuMjEuMjEyNTAyNDkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 15. 15.R Core Team (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL [https://www.R-project.org/](https://www.R-project.org/). 16. 16. Ralph Scherer (2018). PropCIs: Various Confidence Interval Methods for Proportions. R package version 0.3-0. [https://CRAN.R-project.org/package=PropCIs](https://CRAN.R-project.org/package=PropCIs) 17. 17.Victorian COVID-19 data [https://dhhs.vic.gov.au](https://dhhs.vic.gov.au) 29th October, 2020 and Australian Government Department of Health [https://www.health.gov.au/news/health-alerts/novel-coronavirus-2019-ncov-health-alert/coronavirus-covid](https://www.health.gov.au/news/health-alerts/novel-coronavirus-2019-ncov-health-alert/coronavirus-covid)29th October,2020. 18. 18.Lassauniere R, Frische A, Harboe Z et al. Evaluation of nine commercial SARS-C0V-2 immunoassays. medRxiv 20056325 [Preprint]. 10 April 2020 [cited 11 Oct 2020]. Available at: [https://doi.org/10.1101/2020.04.09.20056325](https://doi.org/10.1101/2020.04.09.20056325) 19. 19.Isho B, Abe K, Zuo M, Jamal A, Rathod B et al. Mucosal versus systemic antibody responses to SARS- CoV-2 antigens in COVID-19 patients. medRxiv preprint doi: [https://doi.org/10.1101/2020.08.01.20166553](https://doi.org/10.1101/2020.08.01.20166553). August 29, 2020. \ 20. 20. Lin-Fa Wang, Wan Ni Chi, Chee Wah Tan, Randy Foo et al. Serological differentiation between COVID-19 and SARS infections. Emerging Microbes & Infections 7 July 2020. Vol. 9, 2020 pp. 1497–1505. [https://doi.org/10.1080/22221751.2020.1780951](https://doi.org/10.1080/22221751.2020.1780951) 21. 21. Lin-Fa Wang, Chee Wah Tan, Wan Ni Chia, Mar I-C Chen et al. A SARS-CoV-2 surrogate virus neutralization test (sVNT) based on antibody-mediated blockage of ACE2-spike (RBD) protein-protein interaction. Research Square [https://doi.org/10.21203/rs.3.rs-24574/v1](https://doi.org/10.21203/rs.3.rs-24574/v1) 22. 22.GeurtsvanKessel C, Okba N, Igloi Z et al. An evaluation of COVID-19 serological assays informs future diagnostics and exposure assessment. Nature Communications 2020: 11:3436. [https://doi.org/10.1038/s41467-020-17317-y](https://doi.org/10.1038/s41467-020-17317-y) 23. 23.Okba N, Muller M, Li W, et al. Severe Acute Respiratory Syndrome Coronavirus 2-Specific Antibody Responses in Coronavirus Disease Patients. Emerg. Infect. Dis.2020;26(7): 1478–1488. Available at: [https://dx.doi.org/10.3201.eid2607.200841](https://dx.doi.org/10.3201.eid2607.200841) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F01%2F26%2F2021.01.21.21250249.atom) 24. 24.Elslande J, Decru B, Jonckheere S, et al. Antibody response against SARS-CoV-2 spike protein and nucleoprotein evaluated by four automated immunoassays and three ELISAs. Clinical Microbiology and Infection, [https://doi.org/10.1016/j.cmi.2020.07.038](https://doi.org/10.1016/j.cmi.2020.07.038) 25. 25. Lin-Fa Wang et al. A SARS-CoV-2 surrogate virus neutralization test (sVNT) based on antibody-mediated blockage of ACE2-spike (RBD) protein-protein interaction. Research Square [https://doi.org/10.21203/rs.3.rs-24574/v1](https://doi.org/10.21203/rs.3.rs-24574/v1) 26. 26.Grandjean L, Saso A, Ortiz A et al. Humoral Response Dynamics Following Infection with SARS-CoV-2 medRxiv 20155663. Doi: [https://doi.org/10.1101/2020.07.16.20155663](https://doi.org/10.1101/2020.07.16.20155663) July 22, 2020.