Summary
Here we describe an open and transparent consortium for the rapid development of COVID-19 rapid diagnostics tests. We report diagnostic accuracy data on the Mologic manufactured IgG COVID-19 ELISA on known positive serum samples and on a panel of known negative respiratory and viral serum samples pre-December 2019.
In January, Mologic, embarked on a product development pathway for COVID-19 diagnostics focusing on ELISA and rapid diagnostic tests (RDTs), with anticipated funding from Wellcome Trust and DFID.
834 clinical samples from known COVID-19 patients and hospital negative controls were tested on Mologic’s IgG ELISA. The reported sensitivity on 270 clinical samples from 124 prospectively enrolled patients was 94% (95% CI: 89.60% - 96.81%) on day 10 or more post laboratory diagnosis, and 96% (95% CI: 84.85% - 99.46%) between 14–21 days post symptom onset. A specificity panel comprising 564 samples collected pre-December 2019 were tested to include most common respiratory pathogens, other types of coronavirus, and flaviviruses. Specificity in this panel was 97% (95% CI: 95.65% - 98.50%).
This is the first in a series of Mologic products for COVID-19, which will be deployed for COVID-19 diagnosis, contact tracing and sero-epidemiological studies to estimate disease burden and transmission with a focus on ensuring access, affordability, and availability to low-resource settings.
Introduction
On the 31st of January, six weeks after the first report in Wuhan, China, of SARS-CoV-2, the causative pathogen of Coronavirus Disease 2019 (COVID-19), and 24 hours after the World Health Organization (WHO) declared COVID-19 a public health emergency, the United Kingdom reported its first two imported cases. By the end of April >20,000 hospitalized patients with COVID-19 had died in the UK and >200,000 fatalities have occurred worldwide, with the pandemic spreading to more than 190 countries1. The WHO advises countries to ‘test, test, test’ as one of the pillars for COVID-19 control, although most countries don’t have enough tests for widespread deployment and testing strategies focus on identifying symptomatic patients who need hospitalization or for isolation and use among key workers.2
Current testing is almost exclusively reliant on RT-PCR to detect the virus RNA. These assays are sensitive and specific for SARS-CoV-2, but the scale of the COVID-19 pandemic has caused global shortages of essential reagents for testing, including those for sample collection, RNA extraction and pathogen detection. These have hampered the urgent scale-up of testing required, and has led to the politicization of diagnostic testing with reports of stockpiling resources.3–5 Nucleic acid amplification and detection technologies are generally not deployable at the point-of-need, as most of them require significant laboratory infrastructure, expertise, and skilled staff with appropriate personal protective equipment (PPE).
Governments and industry are increasing testing capacity through the accelerated development of simpler, less elaborate and more affordable rapid diagnostic tests (RDTs). These tests may detect viral protein antigens in nasopharyngeal swabs or oral fluid to determine acute infection, or antibodies in blood to indicate seroconversion after infection, similar to laboratory-based enzyme-linked immunosorbent assay (ELISA) technology. RDTs are more suitable for use at the point-of-need, because although they may offer lower sensitivity and specificity than RT-PCR-based tests in some circumstances, their ease of use and rapid readout of results make them important components of diagnostic and contact tracing control strategies, especially as tools for rapid triage at the frontline of hospitals, primary care facilities, and for reaching into the community and home. Antibody detection will also be key to informing exit strategies for the expected long tail of the pandemic and in supporting surveillance by identifying clusters of infection and convalescent cases or people with milder disease that could seed the infection to generate severe cases.6
The rollout of RDTs is currently marred by a lack of performance data, reference materials, numbers tested and transparency on how tests are being evaluated. Many RDTs have been rushed to market with limited or no independent evaluations of performance. Marketing information is typically based on internal evaluations with poorly designed studies with limited participant characterization. Governments have cancelled orders once independent evaluations have failed to replicate the claims of the manufacturer.7
We describe here a partnership set up to develop low-cost COVID-19 tests to detect (a) SARS-CoV-2 antigens and (b) circulating IgG, IgA and IgM antibodies using lateral flow assays (LFA) delivering results in <15 minutes and (c) complementary ELISA assays for laboratory-based mass screening for seroconversion. Development of these diagnostic tools is a result of a partnership between Mologic (UK), a biotechnology company committed to developing tests for low resource settings at cost of goods in line with Global Access Policy8, the Liverpool School of Tropical Medicine (LSTM, UK), St George’s University of London (SGUL, UK), and the Institut Pasteur de Dakar (IPD, Senegal). Our consortium includes a globally distributed testing network across four continents to evaluate novel RDTs in Malaysia, Kenya, Malawi, China, Spain, North America, and Latin America.
Our aim is to develop COVID-19 RDTs to meet the WHO provisional target product profiles (TPP) for point-of-need tests for antigen and antibody assays9. Antigen assays, could be used for the rapid triage of patients with presumptive COVID-19, to detect asymptomatic or presymptomatic carriers and mild infections for isolation10. Asymptomatic and subclinical infections play an important role in the transmission of SARS-CoV-2 to close contacts, especially among cohabiting family members11–13 and may be responsible for 50%-80% of infections14,15, although, critically, we need accurate tests to confirm this hypothesis.
Furthermore, these point-of-need antigen tests will be formatted as self-tests, offering the additional benefit of being widely used in the home and community settings by professional testers for rapid triage, to test contacts of confirmed cases, and to de-centralise surveillance. Antibody assays have been proposed as part of the ‘exit strategy’ from large scale lockdown of countries as a “back to work” test. However, RDTs remain insufficiently accurate due to their low positive predictive and negative value among low prevalence settings in the community, and because the presence of antibodies have not yet been shown to correlate with long-lasting immunity to reinfection.
Methods
Following the declaration of COVID-19 as a Public Health Emergency of International Concern on 30th January 202016, an application by Mologic, LSTM and SGUL to develop and manufacture rapid tests for COVID-19 was considered successful for funding from the Wellcome Trust and the Department for International Development (DFID) on the 28th of February. The program started on March 6th and the first batch of 15 different prototypes arrived at LSTM and SGUL ready for evaluation two weeks from commencing, and 8 weeks ahead of the official award letter. The first batch of prototypes was evaluated against positive serum obtained from convalescent COVID-19 originally confirmed by Public Health England’s RT-PCR tests and provided by International Severe Acute Respiratory and Emerging Infection Consortium 4C (ISARIC) in the UK, and prospectively collected SGUL in-patient samples. Data from testing were shared and discussed across all partners on the 26th of March. Second, third and fourth rounds of independent iterative validation across three sites were conducted in sequential weeks allowing Mologic to accelerate the best performing prototypes and optimise assay components.
We envisaged this approach that combined the strengths of industry and academia, would speed up the development of these critically needed tests. The subsequent robust and transparent evaluation will provide all stakeholders with accurate information, which is critical for the informed decision of test implementation and their correct placement in the UK’s testing strategy. Prototype RDTs and the ELISA have been evaluated in prospectively collected excess diagnostic waste clinical samples of patients with RT-PCR confirmed COVID-19 at SGUL and a specificity panel from LSTM, Mologic and SGUL of pre-December 2019 bio-banked serum samples.
At SGUL, anonymised excess diagnostic material (and relevant clinical data) from patient samples tested for SARS-CoV-2 in South West London Pathology (SWLP) microbiology laboratory at St George’s, University Hospitals NHS Foundation Trust (SGHFT) were collected. RT-PCR confirmation of SARS-CoV-2 infection in patients from which excess diagnostic material were collected used nasopharyngeal swabs placed into Sigma Virocult®. An aliquot was subsequently placed into Roche PCR lysis buffer for RNA extraction using Roche RNA extraction kits on a Magnapure (with extracts placed into 96 well plates). SARS-CoV-2 detection was undertaken using the Altona Diagnostics RealStar®SARS-CoV-2 RT-PCR Kit (S and E genes).
We have also engaged with the ISARIC 4C consortium, who are providing 50 serum samples on an ongoing basis samples for testing, and we will publish results of these rounds of testing once we reach the sample size.
Ethics
Clinical sample collection was approved at SGUL as part of the DARTS study - IRAS project ID: 282104; REC reference: 20/SC/0171
Results
Here we provide validation data for a novel IgG ELISA for SARS-CoV-2. These data were generated using 96-well plate kits provided by Mologic, in which an early generation assay used NP, S1 and S2 antigens and the finalized version used only NP and S2 antigens (with S1 being removed). Assays were undertaken following the manufacturer’s protocol, initially in triplicate and subsequently in duplicate, with each plate having a diluent (negative), cut off and positive controls (results were considered positive if they were 10% above the cut off value). Data are presented in Tables 1 and 2.
A total of 270 positive samples laboratory confirmed SARS-CoV-2 by RT-PCR, ranging from day 0 to day 42, and 564 negative samples (from 2018–2019) were tested. Clinical samples were analysed from a total of 124 patients, including from 50 women (40.3%) and 74 men (59.7%) with an age range of 26 to 88 years. Overall, the sensitivity was 88% (95% CI: 83.68% - 91.75%) in all samples (238 of 270 positive samples). The specificity was 97% (95% CI: 95.65% - 98.50%) (549 of 564 samples tested).
Sensitivity of the IgG ELISA was 94% (95% CI: 89.60% - 96.81%; 185 of 197 positive samples) among samples greater than 10 from initial laboratory diagnosis. Sensitivity among symptomatic cases was 96% between 14 and 21 days of symptom onset (95% CI: 84.85% - 99.46%) compared with 78% among cases less than 14 days from symptom onset (95% CI: 64.04% - 88.47%).
A total of 209 samples (77.4%) were collected from 90 individuals reporting symptoms of COVID-19 and 61 samples (22.6%) from 34 individuals who were asymptomatic at first swab collection. Of the 34 individuals who were asymptomatic at presentation and positive by RT-PCR, 29 were also COVID-19 positive with IgG ELISA (85.3%). Of the remaining 5 individuals, 3 seroconverted (8.8% of asymptomatic cases on presentation) on repeat testing and 2 individuals remained negative on the IgG ELISA despite repeat testing (5.8% of asymptomatic cases on presentation). Among the 90 RT-PCR positive symptomatic individuals, 11 were IgG ELISA negative at presentation (12.2%), 6 seroconverted on repeat testing (6.7% of symptomatic cases on presentation)., and 5 individuals remained negative on IgG ELISA (5.5% of symptomatic cases on presentation).
Time from swab to laboratory confirmation was comparable between ELISA positive and ELISA negative samples. However, ELISA negative samples were on average 3 days sooner to symptom onset than ELISA positive samples (20.5 days versus 23.5 days, respectively). In addition, swab collection was 6 days earlier among ELISA negative samples than ELISA positive samples (11.6 days versus 17.5 days, respectively) and laboratory confirmation was 5 days earlier (10.1 days versus 15.7 days, respectively). Collectively this suggests a proportion of ELISA negative samples are yet to manifest as positive for IgG detection.
Discussion
Challenges of RDT development
The development of RDTs takes longer than that of RT-PCR tests, which were designed and developed within days of the publication of the first viral genomes17–19, and then validated within weeks20,21. RDTs require the production of viral protein antigen, in the case of antibody detection, or the production of specific antibodies in the case of antigen detection.
Rapid tests are difficult to develop without access to well-characterised patient samples, unlike RT-PCR tests which can be developed using synthetic RNA with the viral target sequence. However, not all RDTs are created equal. In existing RDT markets there is a substantial variation of performance and, for example, the sensitivity of malaria RDTs varies between 31.7% and 100% across manufacturers22. The same variations are documented for Dengue23 and leishmaniasis24, to name but a few diseases. It is thus essential that new COVID-19 tests are subjected to robust independent evaluations, and that results from evaluations are transparent and freely available for scrutiny by governments and the public. The methodology of an evaluation, including the study population, the reference test used, and the handling of the sample have a large bearing on the results of the study. A badly designed study will either under- or overestimate a test’s accuracy, invalidating the decision of whether to implement that particular test or not.
Transparency and Openness
In the UK, the target sensitivity and specificity from the MHRA was published on the 8th April 2020, several days after ELISAs and RDTs were reported to fail evaluations as tests for ‘immunity passports’ and seroprevalence studies.25 The required accuracy for such tests is currently specified at >98% sensitivity and specificity (update 24th April 202026). However, all manufacturers reported lower accuracy levels than this threshold from their own evaluations, and therefore would not have expected to pass these stringent criteria. The current lack of transparency and rapid feedback to product developers removes the opportunity for evaluation data to inform test optimisation and to develop improved products. Additionally, company names were not released with the evaluation data due to non-disclosure agreements25. Not having a TPP by WHO for COVID-19 diagnostics – 3 months since the declaration of a global public health emergency only serves to hamper the accelerated development of novel technologies to support pandemic control. Release of preliminary data to the scientific community for consultation and scrutiny would allow for a wider discussion of results, and a greater understanding of likely test accuracy. Furthermore, we would encourage evaluations within networks in the UK; the greater the sample number, the more accurate predictions are, whereas current MHRA criteria may perversely encourage small, poorly designed studies with large confidence intervals to hit the targets.
As a team of academic evaluators, at the LSTM and St George’s, University of London, and Mologic, a biotechnology company and manufacturer of rapid diagnostic tests, we have sought to aid the development of new RDTs for SARS-CoV-2 antigens and antibodies. We have formed an accelerated validation partnership enabling the independent evaluation of prototype tests, with continual feedback to the test developer allowing for iterative optimisation of the assay. During this process we strive to make results available to the scientific community as rapidly as possible, both via publication in journals, and in frequently updated results summaries on public domains such as BioRxiv (https://www.biorxiv.org) and MedRxiv (https://www.medrxiv.org). This is our first submission of the ELISA IgG data which looks extremely promising and is now available from Mologic with increased manufacturing capability aided by DFID, FIND and BMGF.
Data Availability
All raw data available
Authors’ contributions
ERA, JRAF, HMS, LEC, SK, TP conceived study.
ACA, AJF, GAK, SIO, RLB, GG DJC, BMOD, NME, LH, QH, IM, CMM, HMS, SK performed technical work and handled clinical samples.
JM, KK, LT & TF provided clinical data for samples at LSTM.
YA, MC, DK, SK, TP & HMS developed the DARTS protocol and provided clinical data for samples at SGUL
MC, MD, PD, AD, AJ, KK, ZL, JMK, GNvC, CS, JC, JW, KW, AAS, JRAF developed the ELISA test at Mologic.
ERA, ACA, TE, LEC, SK wrote the first draft of the article.
All authors reviewed the manuscript before submission.
Conflict of Interest
SK is a member of the Scientific Advisory Committee for Foundation for Innovative New Diagnostics (FIND) a not-for-profit that produces global guidance on affordable diagnostics. The views expressed here are personal opinions and do not represent the recommendations of FIND.
Tables and Figures
Funding and Acknowledgements
DFID/Wellcome Epidemic Preparedness coronavirus, 220764/Z/20/Z.
ERA, LEC, LT and TF are affiliated to the National Institute for Health Research Health Protection Research Unit (NIHR HPRU) in Emerging and Zoonotic Infections (NIHR200907) at University of Liverpool in partnership with Public Health England (PHE), in collaboration with Liverpool School of Tropical Medicine and the University of Oxford. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, the Department of Health or Public Health England.
HMS is supported by the Wellcome Trust Institutional Strategic Support Fund (204809/Z/16/Z) awarded to St. George’s University of London.
LT is supported by the Wellcome Trust (grant number 205228/Z/16/Z) We wish to thank the National Association of Blood Bikes (NABB) for their unwavering assistance in transporting samples, prototypes, and validated devices between Liverpool, Bedford, and London. Without the NABB, we could not have delivered two CE marked products in 8 weeks from project launch.