ABSTRACT
Efficient and accurate assays for the differential diagnosis of COVID-19 and/or influenza (flu) could facilitate optimal treatment for both diseases. Diagnostic performance related to SARS-CoV-2 and Flu A/B detection was characterized for the BD SARS-CoV-2/Flu for BD MAX™ System (“MAX SARS-CoV-2/Flu”) multiplex assay in comparison with BD BioGx SARS-CoV-2 Reagents for BD MAX™ System (“BioGx SARS-CoV-2”) and the Cepheid Xpert® Xpress Flu/RSV (“Xpert Flu”). Two hundred and thirty-five nasopharyngeal specimens were obtained from external vendors. MAX SARS-CoV-2/Flu had positive percent agreement (PPA) and negative percent agreement (NPA) values for SARS-CoV-2 and Flu A/B that met FDA-EUA acceptance criteria of >95%.
INTRODUCTION
Since the report of the first cluster of COVID-19 cases in December 2019, over 100 million COVID-19 cases and 2 million COVID-19-related deaths worldwide have been reported by the end of January, 2021, and the numbers continue to rise.[1] In the US, more than 23 million COVID-19 cases and over 450,000 COVID-19 deaths have been recorded through mid-Febuary 2021.[2] Although it is not clear how the 2020-2021 influenza season will impact the health care systems, the 2019-2020 flu season resulted in over 38 million cases involving symptomatic illness and approximately 22,000 deaths in the US.[3] Each year, across the globe, there are an estimated 1 billion cases of influenza, of which, 3-5 million are severe cases and 29,000-655,000 lead to influenza-related respiratory deaths.[4] Although current (as of January 2021) [5] influenza activity is low, this virus has the potential of increasing the workload of healthcare workers already overwhelmed by COVID-19.
Although COVID-19 and influenza spread through a similar mechanism of transmission and have overlapping symptoms, the isolation length and the therapeutic approach for COVID-19 patients and influenza patients are not uniform.[6] The recommended isolation period after symptoms onset is a minimum of 4-5 days [7] for flu, whereas it is a minimum of 10 days for COVID-19,[8] impacting absenteeism and contact tracing. The impact of anti-viral drug therapy, which has traditionally been used for influenza patients, is not well-understood for patients with COVID-19; therefore, safety concerns may preclude any potential efficacy. A similar concern exists for drugs such as remdesivir and corticosteroids, which have been used to treat COVID-19 patients, but are not approved for use in influenza patients, and may (for example, in the case of corticosteroids) have negative side effects in influenza patients.[6] Because the coincidence of high numbers of both COVID-19 and influenza cases in 2020-2021 and in future respiratory virus seasons could produce a significant strain on the healthcare system,[6] differential diagnosis of COVID-19 and influenza, will be an important component for proper patient triage, management, and treatment.
Molecular diagnostics for the detection of SARS-CoV-2, including real-time polymerase chain reaction (RT-PCR) assays, have played an important role in the detection of SARS-CoV-2 and diagnosis of COVID-19 due to their high sensitivity.[9] Similarly, RT-PCR-based detection of Influenza A/B (“Flu A/B”) virus nucleic acid has been established for a number of years and is commonly employed to establish an influenza diagnosis.[10] Due to the expected co-occurrence and potential co-infection of SARS-CoV-2 and Flu A/B, a multiplex RT-PCR assay, for detection all three targets, could help provide faster results and improve patient management and treatment.[11] This report describes the performance of the new BD SARS-CoV-2/Flu assay reagents for BD MAX™ System multiplex assay for detection of SARS-CoV-2 and Flu A/B. The comparator reference methods were the BD BioGx SARS-CoV-2 Reagents for BD MAX™ System and Cepheid Xpert® Xpress Flu/RSV. The objective here was to determine the performance characteristics of the new multiplex BD SARS-CoV-2/Flu assay.
METHODS AND MATERIALS
Specimens and assays
This study, which was conducted as part of a Food and Drug Administration-Emergency Use Authorization (EUA) submission, included data comparing the BD SARS-CoV-2/Flu for BD MAX™ System (“MAX SARS-CoV-2/Flu;” Becton, Dickinson and Company; BD Life Sciences—Integrated Diagnostics Solutions, Sparks, MD, USA) with reference methods, BD BioGx SARS-CoV-2 Reagents for BD MAX™ System (“BioGx SARS-CoV-2;” Becton, Dickinson and Company; BD Life Sciences—Integrated Diagnostics Solutions, Sparks, MD, USA) and Cepheid Xpert® Xpress Flu/RSV (“Xpert Flu;” Cepheid®, Sunnyvale, CA, USA), for detection of SARS-CoV-2 and Flu A/B, respectively. The BioFire® Respiratory 2.1 Panel (“BioFire SARS-CoV-2;” BioFire Diagnostics, Salt Lake City, UT, USA) was used to test specimens for which MAX SARS-CoV-2/Flu and BioGx SARS-CoV-2 provided discrepant results; the cobas® Influenza A/B & RSV assay for use on the cobas® Liat® System (“Liat Flu;” Roche Diagnostics, Indianapolis, IN, USA) was used to test specimens for which MAX SARS-CoV-2/Flu and Xpert Flu provided discrepant results. All assays were performed according to each manufacturer’s instructions for use. MAX SARS-CoV-2/Flu, BioGx SARS-CoV-2, and BioFire SARS-CoV-2 assays were performed at BD Integrated Diagnostics Solutions; Xpert Flu and Liat Flu assays were performed at TriCore Reference Laboratories.
Nasopharyngeal specimens for SARS-CoV-2 testing were obtained from New York Biologics, Inc. (Southhampton, NY, USA) and Trans-Hit Bio (Laval, QC, Canada), and nasopharyngeal specimens for Flu A/B testing were obtained from New York Biologics, Inc (Table S1). Two hundred and thirty-five (235) nasopharyngeal specimens either in Copan Universal Transport Medium (UTM®) or in BD Universal Viral Transport (UVT) system (maintained at −65° C ∼ - 80°C), which were collected between November 30, 2019 to September 3, 2020, were obtained from individuals with ages ranging from ≤5 years of age to ≥60 years of age (Table 1).
SARS-CoV-2 and influenza positivity by reference method or MAX SARS-CoV-2/Flu across age groups
The specimens were collected as part of standard of care (SOC) and the frozen de-identified remnants were used for this research. This article was prepared according to STARD guidelines for diagnostic accuracy studies reporting.
Data analysis
The primary outcome measures for this study were positive and negative percent agreement (PPA and NPA, respectively) point estimates (with 95% confidence interval [95% CI] calculated using the Wilson score method) for the MAX SARS-CoV-2/Flu assay, compared to each respective reference assay. The McNemar test was used for 2×2 classification to test the difference between paired proportions. The calculated difference is that of marginal proportions ([total proportion of SARS-CoV-2, Flu A, or Flu B positives] – [total proportion of positives (for each of the three causes) by clinical diagnosis]). A p-value <0.05 was utilized to distinguish significant differences (note here that a p-value >0.05 indicates only that disagreement between the two diagnostics methods is random). The Cohen’s kappa coefficient was utilized to gauge the agreement between two raters (reference and test) to classify results into mutually exclusive categories. Κ=(Po-Pe)/1-Pe (<0, 0, and >0 indicating agreements worse than, no better or worse than, and better than that expected by chance). Acceptance criteria for the MAX SARS-CoV-2/Flu assay for US FDA-EUA authorization for SARS-CoV-2 was ≥95% for both PPA and NPA. The PPA criteria for Flu A/B was ≥90% (lower bound of the 95%CI ≥80%) and the NPA criteria for Flu A/B was ≥95% (lower bound of the 95%CI ≥90%). Only compliant and reportable results for both MAX SARS-CoV-2/Flu and comparator assays were included in this analysis.
RESULTS
Two hundred and thirty-five (235) specimens were included in this study, from which, three were excluded due to unreportable results from the instrumental faliure. From the remaining 232 specimens, reference method testing for SARS-CoV-2 (BioGx SARS-CoV-2) and Flu A/B (Xpert Flu), resulted in 52 positive SARS-Cov-2 specimens, 59 positive Flu A specimens, and 60 positive Flu B specimens (Table 1). By reference method, 30, 91, and 90 specimens were negative, respectively, for SARS-CoV-2, Flu A, and Flu B. MAX SARS-CoV-2/Flu results were compared to results from each respective reference method to determine PPA and NPA values. MAX SARS-CoV-2/Flu had PPA and NPA values of 96.2% [95%CI: 87.0, 98.9] and 100% [95%CI: 88.7, 100], respectively, for detection of SARS-CoV-2. For Flu A, MAX SARS-CoV-2/Flu had PPA and NPA values of 100% [95%CI: 93.9, 100] and 98.9% [95%CI: 94.0, 99.8], respectively; for Flu B, MAX SARS-CoV-2/Flu had PPA and NPA values of 98.3% [95%CI:91.1, 99.7] and 100% [95%CI: 95.9, 100], respectively (Table 2).
Performance of the MAX SARS-CoV-2/Flu assay for detection of SARS-CoV-2, Flu A and Flu B compared to reference
DISCUSSION
The results here show PPA for the MAX SARS-CoV-2/Flu assay with reference assays meet FDA-EUA acceptance criteria for detection of SARS-CoV-2 (96.2%), Flu A (100%; with a lower bound 95%CI of 93.9%), and Flu B (98.3%; with a lower bound 95%CI of 91.1%). Similarly, compared to reference methods, the MAX SARS-CoV-2/Flu assay was associated with NPA values for detection of SARS-CoV-2 (100%), Flu A (98.9%; with a lower bound 95%CI of 94.0%), and Flu B (100%; with a lower bound 95%CI of 95.9%) that all met FDA acceptance criteria. During discordant testing, the MAX SARS-CoV2/Flu assay was in agreement with the third assays (i.e. BioFire SARS-Cov-2 assay and Liat Flu assay) for both SARS-CoV-2 negative results and for the Flu A positive result by the MAX SARS-CoV-2/Flu assay. For Flu B, the Liat Flu assay agreed with the Xpert Flu assay negative result. However, all discrepant results were associated with high Ct values (ranging from 37.8 to 39.5). The MAX SARS-CoV-2 showed 100% PPA in specimens with reference method results associated with Ct values ≤30 (Table S2). Thus, with its high PPA and NPA for SARS-CoV-2, Flu A, and Flu B, this multiplex assay should reduce specimen collection time and the amount of supplies and reagents necessary to test for both COVID-19 and Flu.
Different approaches are currently available for the detection of SARS-CoV-2 and Flu A/B for the diagnosis of both COVID-19 and influenza, respectively.[10, 12] Although culture-based assays were originally utilized to establish an influenza diagnosis, RT-PCR-based technology for diagnosis of influenza currently represents the laboratory method of choice due to its relatively high analytic and clinical sensitivity, as well as short turn-around time.[13] Likewise, RT-PCR-based assays appear to have higher sensitivity for detection for SARS-CoV-2 compared to culture-based assays.[14] Rapid testing, such as immunochromatic assays used to detect viral antigen, have been developed for detection of both SARS-CoV-2 and Flu A/B.[9, 10] Although rapid tests carry advantages, including decreased time-to-result and ease of implementation in decentralized health care settings, RT-PCR-based assays have increased analytical sensitivity compared to rapid tests.[10] Ultimately, multiple factors should be considered before determining which strategy should be employed. For example, hospitals and their associated laboratory partners, which have established a streamlined workflow and a relatively fast turn-around-time, can effectively employ RT-PCR-based assays—especially for patients admitted and managed according to their symptoms. This strategy carries the benefit of high sensitivity and the ability to rule out etiologic agents with a high degree of assurance.
If the coincidence of high numbers of both COVID-19 and influenza cases occurs during flu seasons, differential diagnosis for the appropriate therapeutic approach could be a challenging feat. Although COVID-19 and influenza spread through a similar transmission mechanism and have overlapping symptomology, specific differences between the diseases do exist. For example, COVID-19 seems to involve a longer time to symptom onset than influenza, and may cause more severe illness in vulnerable populations once symptoms develop.[15] Also, the therapeutic approach for COVID-19 patients and influenza patients is not uniform. The impact of blanket anti-viral drug, which has traditionally been used for influenza patients, is not established as a treatment for SARS-CoV-2-positive individuals. Therefore, safety concerns may preclude any potential efficacy. A similar problem exists for drugs such as remdesivir and corticosteroids.[6] Both medications have been used to treat COVID-19 patients, but are not approved for use in influenza patients, and may (in the case of corticosteroids) have adverse side effects in influenza patients. Distinguishing the diagnosis of COVID-19 and influenza, therefore, will be an important component for proper patient triage, management, and treatment.
Limitations
This research was conducted by using materials obtained from pre-selected frozen remnants, received after routine care. A study involving prospective collection would better inform on the positive and negative predictive values of the assay.
Conclusions
The MAX SARS-CoV-2/Flu assays met US FDA-EUA acceptance criteria for SARS-CoV-2 and Flu A/B detection. Dual detection of the etiologic agents causing COVID-19 and influenza will allow differentiation for those exhibiting common symptoms between the two diseases. This assay should help optimize patient management by decreasing the time and resources required for dual testing. Ultimately, the dual detection method should facilitate an informed decision by physicians on the appropriate treatment for patients exhibiting similar symptoms between the two diseases.
Data Availability
All data are available upon request.
AUTHOR CONTRIBUTIONS
All authors contributed to the interpretation of the data, critically revised the manuscript for important intellectual content, approved the final version to be published, and agree to be accountable for all aspects of the work.
FUNDING
This study was funded by Becton, Dickinson and Company; BD Life Sciences—Integrated Diagnostics Solutions. Non-BD employee authors received research funds to support their work for this study.
POTENTIAL CONFLICTS OF INTEREST
The authors disclose the following conflicts of interest: SP, EL, and CKC are employees of Becton, Dickinson and Company; SY, None.
ACKNOWLEDGEMENTS
We thank the National Institute of Infectious Diseases, Japan, for providing VeroE6TMPRSS2 cells. We also thank Karen Eckert and Karen Yanson (Becton, Dickinson and Company, BD Life Sciences – Integrated Diagnostic Solutions) for their input on the study logistics and content of this manuscript. We thank Yu-Chih Lin and Devin Gary for the editorial assistance. We thank Stanley Chao, Aojun Li, and Yongqiang Zhang (Becton, Dickinson and Company, BD Life Sciences – Integrated Diagnostic Solutions) for statistical support. The individuals acknowledged here have no additional funding or additional compensation to disclose. We are grateful to the study participants who allowed this work to be performed.
