Abstract
Background In October 2020, the National Cancer Institute (NCI) Serological Sciences Network (SeroNet) was established to study the immune response to COVID-19, and “to develop, validate, improve, and implement serological testing and associated technologies.” SeroNet is comprised of 25 participating research institutions partnering with the Frederick National Laboratory for Cancer Research (FNLCR) and the SeroNet Coordinating Center. Since its inception, SeroNet has supported collaborative development and sharing of COVID-19 serological assay procedures and has set forth plans for assay harmonization.
Methods To facilitate collaboration and procedure sharing, a detailed survey was sent to collate comprehensive assay details and performance metrics on COVID-19 serological assays within SeroNet. In addition, FNLCR established a protocol to calibrate SeroNet serological assays to reference standards, such as the U.S. SARS-CoV-2 serology standard reference material and First WHO International Standard (IS) for anti-SARS-CoV-2 immunoglobulin (20/136), to facilitate harmonization of assay reporting units and cross-comparison of study data.
Results SeroNet institutions reported development of a total of 27 ELISA methods, 13 multiplex assays, 9 neutralization assays, and use of 12 different commercial serological methods. FNLCR developed a standardized protocol for SeroNet institutions to calibrate these diverse serological assays to reference standards.
Conclusions SeroNet institutions have established a diverse array of COVID-19 serological assays to study the immune response to SARS-CoV-2 virus and vaccines. Calibration of SeroNet serological assays to harmonize results reporting will facilitate future pooled data analyses and study cross-comparisons.
Introduction
The National Cancer Institute (NCI) Serological Sciences Network for COVID-19, or SeroNet, was launched on October 8, 2020, as a collaborative initiative to expand research on immune responses to SARS-CoV-2. SeroNet is comprised of investigators from 25 US biomedical research institutions, working in partnership with the Frederick National Laboratory for Cancer Research (FNLCR) and the SeroNet Coordinating Center, which is managed by the FNLCR.1 Of the 25 participating research institutions, 8 are designated as Serological Sciences Centers of Excellence (funded by U54 grants), 13 are funded with U01 grants to carry out specific research projects related to COVID-19 immunity, and 4 institutions are funded by subcontracts and are designated as Serological Sciences Network Capacity Building Centers.1
One of the primary goals of this partnership is “to develop, validate, improve, and implement serological testing and associated technologies.”1 To this end, SeroNet formed a working group, the Serology Assays, Samples, and Materials Operations Group (abbreviated as “Serology Assay Ops”), in December 2020 to allow for coordinated development and collaborative sharing of serology assay procedures, and to establish processes for harmonizing and standardizing methodologies using reference materials across institutions. Establishing harmonized and standardized SARS-CoV-2 serological assays can allow cross-comparison and pooling of research study results and facilitate clinical interpretation of results for patient care.
While there are 85 serological assays approved by the FDA for emergency use,2 the quick development of assays has led to the lack of harmonized cut-offs and reporting units. Furthermore, there are no consensus guidelines on reporting standards or clarity on the clinical interpretation and relevance of results. This has created a complex landscape for interpreting both research and clinical serological assay results. For example, several studies have reported on heterogeneity in serological assay performance that would have a significant impact on research study conclusions and clinical interpretations related to longitudinal serosurveillance.3-6 Specifically, certain assays demonstrate reduced sensitivity over time after an initial SARS-CoV-2 infection diagnosis. Muecksch et al. reported that the Abbott SARS-CoV-2 anti-Nucleocapsid IgG assay dropped from a peak sensitivity of 98% at 21 – 40 days post-PCR diagnosis, to around 70% when patients were tested ≥ 81 days post-diagnosis, whereas the Roche Elecsys SARS-CoV-2 anti-Nucleocapsid total antibody assay and Siemens SARS-CoV-2 anti-receptor-binding domain (RBD) total antibody assay both maintained high sensitivity (95 – 100%) on the same set of serial samples. Narowski et al. also found a significant decline in the longitudinal sensitivity of their lab-developed nucleocapsid assay in a study of healthcare workers.6 Perez-Saez et al. similarly demonstrated that the rates of sero-reversion at least 8 months after the initial infection differed greatly depending on the serological assay used.4 While the sero-reversion rate of the EuroImmun semiquantitative anti-S1 IgG ELISA was 26%, the rate was significantly lower for the Roche anti-Nucleocapsid total antibody assay (1.2%) and the Roche semiquantitative anti-RBD total antibody assay (0%).4 Additionally, numerous studies rely on neutralization assays as gold standard methods for determining the functional relevance of ligand-binding methods, but comparison studies have demonstrated variability in results for live-virus neutralization, pseudovirus neutralization, and surrogate neutralization assays (e.g., ACE2 inhibition assays),7-9 raising the importance of assay harmonization and standardization across laborartories.
Therefore, SeroNet aims to address these knowledge gaps in SARS-CoV-2 serological assay research by establishing collaborative initiatives to characterize, compare, and harmonize SARS-CoV-2 serological assays. This manuscript describes the depth and breadth of serological assays developed and implemented within the SeroNet consortium, and outlines a proposed process to establish assay traceability to the U.S. SARS-CoV-2 serology standard reference material and to the WHO International Standard (WHO IS 20/136) for these diverse assays, with the ultimate goal of establishing harmonized reporting standards. This will facilitate cross-comparison of results and provide clarity for their clinical interpretation, including in response to circulating SARS-CoV-2 variants.
Methods
Compilation of data on SeroNet serological assays
SeroNet institutions were queried by email between January and July 2021 and asked to complete a comprehensive serological assay survey to describe serological assays developed or implemented at their institution. The survey requested information on assay and sample type(s), instrument platform and reagents, data output, antibody isotype(s) detected, targeted antigens and virus strain(s), assay performance, cut-offs, use of standards and quality controls, method comparison studies, regulatory status, current use/applications for assays, and publications using each assay.
Protocol for establishing traceability of serology assays to the U.S. SARS-CoV-2 serology standard and First WHO International Standard for anti-SARS-CoV-2 immunoglobulin
FNLCR developed a recommended protocol for SeroNet institutions to establish serology assay traceability to the U.S. SARS-CoV-2 Serology Standard. In short, for enzyme-linked immunosorbent assay platforms (ELISA), the U.S. SARS-CoV-2 standard is measured on the same 96-well plate as the daily assay standard, run as serial dilutions in triplicate and quadruplicate respectively (Figure 1). Standard curves are constructed for both the U.S. SARS-CoV-2 Serology standard and daily assay standard. A test of parallelism and linearity between the two dose-response curves is then performed to ensure that immunoaffinity differences or matrix effects do not prevent accurate calibration with the U.S. SARS-CoV-2 Serology Standard. Units based on the U.S. SARS-CoV-2 serology standard can then be assigned to the assay daily standard, to harmonize assays and units for results reporting. For non-plate-based assay platforms, similar dilution-based standard curves are constructed.
Example plate map for assay calibration set-up
Numbers indicate suggested serial dilutions. Serial dilutions of primary and secondary calibrators (reference materials) are plated in triplicate, and the daily internal assay standard is plated in quadruplicate.
C_STD: Daily internal assay standard
STD-C1, C2, and C3: Primary calibrator (primary reference material or standard)
STD-T1, T2, and T3: Secondary calibrator (secondary reference material or standard)
NEG: Negative control sample
PC1: Positive control sample 1
PC2: Positive control sample 2
Traceability of the FNLCR standard to the First WHO International Standard (IS) for anti-SARS-CoV-2 immunoglobulin (20/136) was established, to allow SeroNet assays to convert U.S. Serology Standard units to WHO IS units. The WHO IS 20/136 is a freeze-dried equivalent of 0.25 mL of pooled plasma from 11 individuals with a history of SARS-CoV-2 infection. Once reconstituted the WHO standard has an arbitrary unitage of 1000 binding antibody units (BAU)/mL. Eight serial dilutions of the U.S. SARS-CoV-2 serology standard and WHO IS 20/136 were run in triplicate. Parallel line analysis, which included tests for parallelism and linearity, was utilized to assign WHO IS 20/136 standard units to the U.S. SARS-CoV-2 serology standard; this will allow SeroNet institutions to convert U.S. SARS-CoV-2 serology standard units to WHO standard units for serological methods.
Results
SeroNet Serology Assay data
Of the 25 institutions involved with SeroNet, 23 institutions reported performing between one to seven serology assays, and provided descriptive and performance data. Serology assay data were also obtained from the Frederick National Laboratory for Cancer Research (FNLCR) and National Institute of Standards and Technology (NIST), both of which collaborate with SeroNet. Collectively, Seronet institutions reported development of 27 in-house ELISA methods (Table 1).6,10-26 The majority of ELISA methods were developed for testing of serum and/or plasma, with additional methods available for testing dried blood spots (DBS), saliva/oral fluid, and breast milk. Two methods have been granted FDA EUA approval, 3 methods are pending FDA EUA, 4 methods are validated for high-complexity testing in a CLIA-certified laboratory, and 18 methods are for research-use only (RUO). Diagnostic sensitivity and specificity for in-house ELISA methods ranged from 67.4 – 100 % and 90 – 100%, respectively.
Laboratory-developed singleplex ELISA assays
Eight institutions reported development or use of multiplex or protein arrays for antibody detection (Table 2).27-37 Sample types include serum, plasma, DBS, saliva, and bronchoalveolar lavage (BAL) fluid. Diagnostic sensitivity and specificity for multiplex and protein array methods range from 85 – 98.8 % and 95.2 – 100 %, respectively. Neutralization assays were developed by 9 institutions, with sample types including serum, plasma, BAL fluid, nasal wash, DBS, and breast milk (Table 3).15,24,29,38-50 Assays fall into three mechanistic categories – competitive binding assays, pseudotyped neutralization assays, and live virus neutralization assays. The competitive binding assay measures the ability of antibodies to block interactions between the SARS-CoV-2 receptor binding domain and human ACE2 receptor. Virus pseudotype neutralization assays, mainly HIV- and VSV-based, use full length spike incorporated in the viral particle to measure the capability of neutralizing antibodies to block viral entry into the target cells. SARS-CoV-2 live virus plaque or focus reduction neutralization assays measure the ability of neutralizing antibodies to block the spreading infection of authentic SARS-COV-2 in cell culture. Diagnostic sensitivity and specificity for neutralization methods developed within SeroNet range from 93 – 100 % and 97 – 100 %, respectively. Lastly, 9 institutions report use of 12 commercial serology methods (Table 4). Commercial methods detect IgG, IgM, and/or total Ig to spike, RBD, and/or nucleocapsid antigens in serum or plasma. Of the commercial methods in use, 10 are FDA EUA approved, 1 is pending FDA EUA, and 1 is RUO.
Laboratory-developed multiplex assays
Neutralization assays
Commercial assays
Establishment of SeroNet assay traceability to the U.S. SARS-CoV-2 Serology Standard and First WHO International Standard for anti-SARS-CoV-2 Immunoglobulin
Units for the U.S. SARS-CoV-2 Serology standard were initially established by FNLCR based on measurements performed by eight laboratories (Table 5). Subsequently, FNLCR further established traceability of the U.S. SARS-CoV-2 Serology standard to the WHO IS 20/136 by using four FNLCR ligand binding serology assays, with assessment of neutralization tested at NIAID’s Integrated Research Facility (IRF) (Table 5). The U.S. SARS-CoV-2 serology standard was made available to the public in December 2020. Thus far, there have been 124 requests for U.S. SARS-CoV-2 standard material, and 19 requests for the reference panel samples.
Units assigned to the U.S. SARS-CoV-2 Serology Standard
Discussion
SeroNet has collectively established a diverse array of methodologies for measurement of SARS-CoV-2 antibodies in a variety of biological fluids. Methods include laboratory-developed ELISAs, multiplex assays, and neutralization assays, most used for research-only purposes, as well as commercial assays available for patient care or research studies. Assays have been developed to test unique sample types, including DBS, saliva/oral fluid, breast milk, nasal washes, and bronchoalveolar lavage fluid. Binding assays identify IgM, IgG, IgA, and/or total antibodies to nucleocapsid, spike, RBD and/or N-terminal domain (NTD) antigens, and neutralization assays rely on three methods to quantify antibodies with functional neutralizing activity. This diversity of assay methods allows for robust investigation of multiple aspects of the serological response to SARS-CoV-2 infection and vaccination, and for cross-comparison of assay performance across platforms and institutions within SeroNet.
With the rapid development of numerous methods for serological assessment, as exemplified by the depth and breadth of assays within SeroNet, it is critical to establish assay harmonization and standardized reporting units to facilitate cross-comparison of results across studies, as well as for streamlined meta-analyses. To this end, FNLCR has provided the U.S. SARS-CoV-2 serology standard reference material, which has traceability to the First WHO International Standard for anti-SARS-CoV-2 Immunoglobulin, to SeroNet sites performing serological assays, to allow establishment of standardized reporting of results in binding antibody units (BAU) per mL traceable to the WHO standard. These efforts may more rapidly facilitate the establishment of a universal cut-off as a correlate of protection, which will be critical to broaden the clinical utility of serological testing for patient care, will allow vaccine trials to transition to an immunogenicity endpoint rather than morbidity or mortality endpoints (immuno-bridging), and will guide decisions regarding optimal scheduling of future vaccine doses to optimize protective efficacy for the general immunocompetent population and susceptible immunocompromised sub-populations.
In summary, SeroNet is well-positioned to rapidly and collaboratively advance our understanding of the immune response to both SARS-CoV-2 infection and vaccination, with ongoing evaluation of serological responses to SARS-CoV-2 variants of concern. The collective effort of institutions involved with SeroNet, to both establish diverse and complementary serological assays, and establish traceability of these diverse assays to the WHO standard, will allow for comprehensive investigation of immune responses and facilitate pooled analyses within the SeroNet consortium. This will enable achievement of the ultimate goal – establishment of a universal correlate of protection cut-off, which will provide a foundation for broader clinical use of serologic testing, as a guide for future decisions on scheduling of COVID-19 vaccine boosters, as well as for general assessment of COVID-19 vaccine immune responses against vaccine viruses and newly evolving variants of concern.
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
Funding sources
Funded by NCI Contract No. 75N91019D00024, Task Order No. 75N91021F00001, award numbers 21X089 (J.L., V.M., J.P., J.Q., L.S.), 21X090 (J.M.C., N.C.O.), 21X091 (A.B.K., S.N.T., B.T.), and 21X092 (C.C-C., A.F-B., F.K., D.R.M., V.S., A.W.); and NCI Grants U54CA260591 (K.S.), U01CA260469 (T.L., D.A.G., S.W.G., C.D.H., K.K., N.P.), U54CA260543 (L.P.), U54CA260582 (S-L.L., G.L.), U54CA260492 (S.L.K., S.D.), U54CA260563 (F.E-H.L., M.S.S., N.S.H., J.L.D., J.D.R.), U01CA260541 (J.D.B., A.K.P., T.L.S., E.T.S., C.A.S., P.P., A.M.E.), U01CA260526 (K.W.B., J.C.F., J.L.K.), U01CA261276 (R.A.B., C.F., A.M.M.), U01CA260539 (C.L.K.), U01CA260508 (L.M.S., A.P.D., R.C.G., D.T.H., W.T.L., J.L.Y., A.F.P.), U01CA260462 (S.B., S.P.).
Potential conflicts of interest
A.B.K. is a consultant for Roche Diagnostics and has received research support from Siemens Healthcare Diagnostics and Kyowa Kirin Pharmaceutical Development. J.D.B., A.K.P., E.T.S., and T.L.S. have received research support from Altimmune. J.D.R. and M.S.S. are co-inventors on a patent filed by Emory University covering the serology assay described in this manuscript. M.S.S. serves on the advisory board for Moderna and Ocugen. F.E.L. is the founder of MicroB-plex, Inc. J.L.D. is the CSO of MicroB-plex, Inc. N.S.H. has been a senior scientist at MicroB-plex, Inc. F.E.L. has research grants from the Gates Foundation and Genentech, is on the SAB of Be Biopharma, Inc., and received royalties from BLI, Inc., as an inventor for the plasma cell survival media. S.B. has research support from Merck and Pfizer, and is a member of the CMV Vaccine Advisory Committees of Merck and Moderna. S.P. has research support from Moderna. D.A.G. is the Chief Scientific and Strategy Advisor of Salimetrics, LLC. S.W.G. is the Chief Scientific Officer of Salimetrics, LLC. Mount Sinai has licensed serological assays to commercial entities and has filed for patent protection for serological assays. The Icahn School of Medicine at Mount Sinai has filed patent applications relating to the COVID-19 serological assay (“Serology Assay”) and NDV-based SARS-CoV-2 vaccines which list F.K. (“Serology Assay”, vaccines), V.S. (“Serology Assay”), A.F-B. (“Serology Assay”), D.R.M. (“Serology Assay”), and C.C-C. (“Serology Assay”) as co-inventors. The foundational “Serology Assay” intellectual property (IP) was licensed by the Icahn School of Medicine at Mount Sinai to commercial entities including Kantaro Biosciences, a company in which Mount Sinai has a financial interest. All remaining authors report no relevant conflicts of interest.
Acknowledgments
Footnotes
Co-authors inadvertently left off the initial submission were added, along with their funding sources and disclosures. Initials of authors added next to funding sources for clarity.
References
