Abstract
Repeating the BinaxNOW antigen test for SARS-CoV-2 by two groups of readers within 30 minutes resulted in high concordance (98.9%) in 2,110 encounters. BinaxNOW test sensitivity was 77.2% (258/334) compared to real-time reverse transcription-polymerase chain reaction. Repeating antigen testing on the same day did not significantly improve test sensitivity while specificity remained high.
Manuscript
Strategies to curb the current coronavirus disease 19 (COVID-19) pandemic are increasingly reliant on antigen-based diagnostics because of low cost, availability, and rapid turn-around. Abbott’s BinaxNOW COVID-19 Ag Card (here on referred to as BinaxNOW) is a lateral flow antigen test resulting in 15 minutes and was initially approved for use in symptomatic individuals [1]. Recent reports suggest lower sensitivity compared to real-time reverse transcription-polymerase chain reaction (RT-PCR) among asymptomatic individuals [2-5]. The ideal frequency of serial rapid tests and the role of confirmatory rapid testing at the same encounter is not well-described. To evaluate repeat antigen testing and test performance, this investigation embedded repeat BinaxNOW testing and simultaneous RT-PCR confirmation at a testing site where antigen tests were freely available to the public regardless of symptoms or exposures [6]. Concordance of repeat BinaxNOW testing as well as test performance in a community setting with high prevalence of SARS-CoV-2, the virus that causes COVID-19, is described.
Methods
Participants were recruited from registrants at a community SARS-CoV-2 testing site in Oshkosh, Wisconsin, during an ongoing mass surge testing campaign. The surge testing campaign was sponsored by the Wisconsin Department of Health Services, University of Wisconsin, and the U.S. Department of Health and Human Services [6]. For this investigation, individuals were recruited to undergo two BinaxNOW tests and an RT-PCR test regardless of their symptom status or result of the initial BinaxNOW test. The first BinaxNOW test was done by trained staff from the routine testing site while the second BinaxNOW test was done by a separate group of trained CDC staff, both according to manufacturer’s instructions. All swabs were supervised, self-collected, and from the anterior nares. Participants completed a questionnaire on demographics, exposures, and symptoms. Approximately 30 minutes after the initial swab was taken, each participant provided documentation of their initial BinaxNOW test result and two additional self-collected swabs were taken by CDC staff. Participants were instructed to simultaneously insert one swab into the left nostril and one swab into the right nostril, rotate five times, swap nostrils, and rotate five times again. One swab was used to perform the second BinaxNOW test immediately, and the other swab was placed in viral transport medium and transported to the Marshfield Clinical Research Institute laboratory for RT-PCR testing within two to five days. BinaxNOW results were considered invalid/indeterminate if no lines were seen in the results window or if only the sample line was seen. A three viral target RT-PCR assay (S gene, N gene, Orf1Ab) for SARS-CoV-2 was conducted. Positive specimens were defined as having at least two targets with a threshold cycle (Ct) value ≤37 per manufacturer’s instructions [7]. MagMAX Viral/Pathogen Nucleic Acid Isolation Kit (REF A48310) was used for RNA extraction. Specimens with inconclusive results (defined as one of three positive targets) were re-tested. Symptomatic participants were defined as reporting one or more of 15 symptom criteria at enrollment [8]. RT-PCR was the gold standard for defining antigen test performance. Ninety-five percent confidence intervals (CI) for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated using the Clopper-Pearson method. The Mann-Whitney-U test was used to test rank differences for Ct values; chi-square tests were used to test for differences in categorical variables; and t-tests were used to test for differences in ages. Statistical significance was defined as p<0.05. Statistical analysis was performed using R 1.3.1056.
Results
Between November 16, 2020, and December 15, 2020, data on 2,127 participant encounters were collected, capturing 22% of routine tests performed at the community testing site. Six inconclusive RT-PCR results, seven missing RT-PCR results, and four indeterminate BinaxNOW test results from separate encounters were excluded for a total of 2,110 participant encounters with all tests (two BinaxNOW and one RT-PCR) among 2,024 unique individuals. Children under the age of 18 years provided 10.7% (225/2,110) of specimens (Supplementary Table 1).
Test positivity was 12.5% (265/2,110) for the initial BinaxNOW test, 12.7% (269/2,110) for repeat BinaxNOW test, and 15.8% (334/2,110) for RT-PCR. Of the 334 RT-PCR positive specimens, the N gene target Ct values for 258 specimens with positive initial BinaxNOW tests were significantly lower compared to 76 specimens with negative initial BinaxNOW results (20.4 vs 29.8, p<0.01, Supplementary Figure 1). Addition of a second BinaxNOW led to less than 1% increase in the combined percent positivity of BinaxNOW tests (279/2,110, 13.2%). The two sequential BinaxNOW tests were 98.9% concordant (2,086/2,110), with 255 concordant-positive and 1,831 concordant-negative pairs. There were 24 encounters with discordant BinaxNOW results (Table 1) of which 19 (79.2%) were RT-PCR positive.
The overall sensitivity of the initial BinaxNOW test compared to RT-PCR was 77.2% (258/334 95%CI 72.4–81.6), specificity was 99.6% (1,769/1,776, 95%CI 99.2– 99.8), PPV was 97.4% (258/265, 95%CI 94.6–98.9), and NPV was 95.8% (1,769/1,845, 95%CI 94.9–96.7) (Table 1). Among symptomatic individuals, the sensitivity of a single BinaxNOW test was 78.6% (221/281, 95%CI 73.4–83.3). In individuals within seven days of symptom onset, sensitivity was 81.9% (199/243, 95%CI 76.5–86.5). Among individuals reporting no current symptoms, the sensitivity was 68.8% (33/48, 95%CI 53.7–81.3). Repeating a second BinaxNOW test resulted in a sensitivity of 81.4% (272/334, 95%CI 76.8–85.5, Table 1). Asymptomatic antigen positive participants had a higher false positive proportion (5/38, 13.2%) compared to symptomatic antigen positive participants (2/223, 0.9%).
Discussion
In this investigation of the BinaxNOW test, the overall sensitivity and sensitivity compared to RT-PCR among people with symptom onset within seven days were consistent with performance reported to the Federal Drug Administration (FDA) by the manufacturer [1]. Among asymptomatic individuals, sensitivity was lower consistent with other reports[2-4]. The reproducibility of the BinaxNOW test in this community setting by two separate groups of testers and individual readers was high. Only 1.1% of paired specimens tested within approximately 30 minutes of each other had differing BinaxNOW test results. When BinaxNOW tests were discordant, the RT-PCR was usually positive. Addition of a second BinaxNOW test at the same encounter did not significantly improve test sensitivity and offers low yield for capturing additional COVID-19 cases in this setting. Further work is needed to assess other antigen test combinations using a repeated test strategy and to identify the ideal frequency of repeat BinaxNOW testing.
This investigation supports foregoing confirmatory RT-PCR testing in symptomatic antigen test positive individuals per current guidelines, given <1% false positive BinaxNOW test results among symptomatic cases [9]. The FDA has warned healthcare workers of the potential for false positive antigen tests particularly in low-prevalence settings [10]. The BinaxNOW had overall high specificity regardless of symptom status (>99%) and high positive predictive value (97.4%) in a population with high prevalence (15.8%). Prevalence among asymptomatic individuals was lower (5.8%), but this population also had a relatively high positive predictive value (86.8%). In moderate-to-high pretest probability settings, foregoing a RT-PCR confirmatory test in asymptomatic antigen-positive individuals could be considered if resources are limited. The trade-off would be a relatively small but non-trivial increase in false positive results and the associated consequences (e.g., missed work/school, increased stress, unnecessary contact tracing).
False negative antigen tests are often consequential given the risk of furthering transmission due to perceived lack of infection. The significantly higher RT-PCR Ct values among BinaxNOW antigen-negative individuals suggest that nasal specimens from this group had less viral RNA (and possibly lower viral load) compared to antigen positive specimens. Others have shown reduced sensitivity of BinaxNOW antigen testing in asymptomatic individuals, and one explanation is that this group may be in the early infectious period or recovery period when viral load is lower [5, 11, 12].
There are several limitations to this investigation. The population was largely white, non-Hispanic, and older, and the findings may not be generalizable to other settings. Lack of swabbing supervision in other settings could alter test performance. Among asymptomatic individuals, a larger sample size is needed for more precise sensitivity estimates. RT-PCR was used as the gold standard for SARS-CoV-2 detection; however, since post-infectious individuals recovering from COVID-19 may have prolonged detectable viral shedding, the performance of BinaxNOW assays for detecting contagiousness may vary from these results. This investigation is specific to the BinaxNOW test, and the findings cannot be applied directly to other antigen tests.
This investigation provides BinaxNOW test performance in a real-world community setting. The BinaxNOW test exhibits minimal user error and high concordance (98.9%) when repeated at the same encounter offering low yield for capturing additional COVID-19 cases. Symptomatic BinaxNOW positive individuals in moderate-to-high pretest probability situations do not routinely need RT-PCR confirmatory tests because of high test specificity. In certain settings with high prevalence and limited resources, it may also be reasonable to forego RT-PCR confirmation in asymptomatic BinaxNOW positive individuals. Symptomatic negative individuals should continue isolation until RT-PCR confirmation. Quarantine should continue to be emphasized for asymptomatic BinaxNOW negative individuals after a close contact exposure. Given the costs and turn-around time for receiving RT-PCR results, antigen tests for SARS-CoV-2 are invaluable tools to break chains of transmission. Identification of the ideal frequency of antigen testing based on exposures, symptoms, RT-PCR turn-around times, and local incidence levels is needed.
Data Availability
Data will be made available upon reasonable request.
Funding
This work was funded by the Centers for Disease Control and Prevention.
Disclaimer
The findings and conclusions in this report are those of the author(s) and do not necessarily represent the official position of the Centers for Disease Control and Prevention (CDC). Use of trade names and commercial sources is for identification only and does not imply endorsement by the U.S. Department of Health and Human Services or CDC.
Potential conflicts of interests
None of the authors reported conflicts of interests. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interests.
Acknowledgments
The authors thank the participants in Wisconsin for contributing to this investigation. This investigation was possible due to the vision of University of Wisconsin leadership including Tommy Thompson (President, University of Wisconsin System), Ray Cross (President Emeritus, University of Wisconsin System), Scott Neitzel (Vice President University Relations, University of Wisconsin System), Andrew J. Leavitt (Chancellor, University of Wisconsin Oshkosh), Stefan Fletcher (Director, Administrative Policies and Special Projects, University of Wisconsin System), Stacey Rolston (Interim Executive Director of UW-Shared Services, University of Wisconsin System), and Jo Carter (Project Portfolio Manager, University of Wisconsin System) as well as the U.S Department of Health and Human Services community-based testing program. The authors thank Ramika Archibald and Mary Wedig for assistance with reporting.