Abstract
Background Performance of Rapid Antigen Tests for SARS-CoV-2 (Ag-RDT) varies over the course of an infection, and their performance is not well established among asymptomatic individuals.
Objective Evaluate performance of Ag-RDT for detection of SARS-CoV-2 in relation to onset of infection for symptomatic and asymptomatic participants.
Design, Setting, and Participants Prospective cohort study conducted from October 2021 to February 2022 among participants > 2 years-old from across the US who enrolled using a smartphone app. During each testing encounter, participants self-collected one nasal swab and performed Ag-RDT at home; at-least fifteen minutes later, a second nasal swab was self-collected and shipped for SARS-CoV-2 RT-PCR at a central lab. Both nasal swabs were collected 7 times at 48-hour intervals (over approximately 14 days) followed by an extra nasal swab collection with home Ag-RDT test 48-hours after their last PCR sample. Each participant was assigned to one of the three emergency use authorized (EUA) Ag-RDT tests used in this study. This analysis was limited to participants who were asymptomatic and tested negative by antigen and molecular test on their first day of study participation.
Exposure SARS-CoV-2 positivity was determined by testing a single home-collected anterior nasal sample with three FDA EUA molecular tests, where 2 out 3 positive test results were needed to determine a SARS-CoV-2 positive result. Onset of infection was defined as day on which the molecular PCR comparator result was positive for the first time.
Main Outcomes and Measures Sensitivity of Ag-RDT was measured based on testing once (same-day), twice (at 48-hours) and thrice (at 96 hours). Analysis was repeated for different Days Post Index PCR Positivity (DPIPP) and stratified based on symptom-status on a given DPIPP.
Results A total of 7,361 participants enrolled in the study and 5,609 were eligible for this analysis. Among 154 eligible participants who tested positive for SARS-CoV-2 infection based on RT-PCR, 97 were asymptomatic and 57 had symptoms at onset of infection (DPIPP 0). Serial testing with Ag-RDT twice over 48-hours resulted in an aggregated sensitivity of 93.4% (95% CI: 89.1-96.1%) among symptomatic participants on DPIPP 0-6. Among the 97 people who were asymptomatic at the onset of infection, 19 were singleton RT-PCR positive, i.e., their positive test was preceded and followed by a negative RT-PCR test within 48-hours. Excluding these singleton positives, aggregated sensitivity on DPIPP 0-6 for two-time serial-testing among asymptomatic participants was lower 62.7% (54.7-70.0%) but improved to 79.0% (71.0-85.3%) with serial testing three times at 48-hour interval.
Discussion Performance of Ag-RDT within first week of infection was optimized when asymptomatic participants tested three-times at 48-hour intervals and when symptomatic participants tested two-times separated by 48-hours.
Question What is the performance of serial rapid antigen testing (Ag-RDT) in the first week of infection among symptomatic and asymptomatic SARS-CoV-2 infections?
Findings Serial testing with Ag-RDT two-times separated by 48-hours resulted in detection of more than 90% of SARS-CoV-2 infections when symptomatic participants began testing within first week from onset of molecular positivity; participants who were asymptomatic when they began testing within the first-week of molecular positivity observed a sensitivity of 79.0% when they performed three rapid antigen-tests, 48 hours apart.
Meaning To optimize detection of SARS-CoV-2 infection with home antigen tests, people suspected to be infected with SARS-CoV-2 virus should test twice at least 48-hours apart if they are symptomatic and three times at 48-hour intervals if they do not have symptoms (asymptomatic).
Comparator positive composite definition of molecular positivity if majority of molecular assays were positive
Days Past Index Comparator Positive (DPIPP) Number of calendar-days past the day when first Comparator positive was observed
Onset of Infection DPIPP 0, when first Comparator positive was observed
Symptomatic and Asymptomatic Cases Based on presence or absence of self-reported symptoms on the day of testing. Sensitivity was measured for Symptomatic and Asymptomatic cases on DPIPP 0-10
First week of Infection DPIPP 0 - 6
Introduction
SARS-CoV-2 diagnostic testing remains a cornerstone in our nation’s fight against COVID-19, and at-home rapid antigen tests (Ag-RDTs), while not perfect, provide a fast and convenient testing option. This type of test is available without a prescription (i.e., over-the-counter [OTC]), easy-to-use, widely available, and, in some cases, preferred by the population over molecular assays that require appointments, waiting in-line at testing centers, and waiting 24-48 hours for results.1–3 Despite their popularity, key gaps remain in our understanding of these tests, notably their performance among asymptomatic people. Reports on Ag-RDT performance among asymptomatic individuals have been highly varied, ranging from sensitivities of 35.8% to 71% in cross-sectional screening evaluations.4,5 However, few studies have evaluated serial testing performance of Ag-RDTs among asymptomatic individuals. Furthermore, emergency use authorization (EUA) holders of OTC antigen tests are responsible for demonstrating Ag-RDT performance in a population with asymptomatic infection using serial testing. This manuscript describes primary findings from a large study designed in coordination with the National Institutes of Health (NIH), Food and Drug Administration (FDA), and three major rapid antigen test manufacturers to evaluate the performance of serial testing using rapid antigen tests for detection of SARS-CoV-2 among asymptomatic individuals within the first week of infection.
Methods
Study Population and Design
This prospective cohort study enrolled participants between October 18, 2021 and January 31, 2022 over the age of two years from across the country through a novel digital siteless study protocol. Details of the study design and protocol are described elsewhere.6 This study was approved by WIRB-Copernicus Group (WCG) Institutional Review Board (20214875). In brief, participants were eligible to enroll through a smartphone app if they did not have a SARS-CoV-2 infection in the prior three months, were without any symptoms in the 14 days prior to enrollment and were able to drop-off prepaid envelopes with nasal swab samples at their local FedEx drop-off location. Enrolled participants were assigned to one of three types of EUA Ag-RDT (Quidel QuickVue At-Home OTC COVID-19 Test, BinaxNow COVID-19 Antigen Self-Test, or BD Veritor At-Home COVID-19 Test) and received a home delivery of 10 Ag-RDTs and 7 home collection kits for reverse transcriptase polymerase chain reaction (RT-PCR) samples. Participants were asked to perform two self-collected bilateral anterior nasal swab collections and paired testing (Ag-RDT (at home) and RT-PCR (mailed to central lab)) every 48-hours for 7 times over a two-week period, with an additional end of study bilateral anterior nasal swab collection with home Ag-RDT test 48-hours after the last paired test. Two FDA-authorized, high sensitivity RT-PCR assays were performed on each nasal swab sample received at the central lab, and an additional tiebreaker assay was performed if the two RT-PCR assays were discordant.
Measures
Ag-RDT results were based on self-report (Quidel QuickVue At-Home OTC COVID-19 Test, BinaxNow COVID-19 Antigen Self-Test) or automatic reader (BD Veritor At-Home COVID-19 Test), as per EUA instructions for use. Molecular comparator RT-PCR results were based on a combination of molecular test results for detection SARS-Cov-2 infection (Supplementary Table 1), and onset of infection was defined as day on which the molecular comparator result was positive for the first time. To approximate performance of Ag-RDT if a person starts testing on different from onset of infection, we identified Days Past Index PCR Positivity (DPIPP) as different strata, for which, performance was calculated. Symptomatic or asymptomatic classification was based on presence or absence of symptoms on DPIPP for which the performance is calculated. Therefore, an individual who was asymptomatic on DPIPP 0 may become symptomatic on DPIPP 2 and vice-versa.
Statistical Analysis
Participants were eligible for inclusion in this analysis if they did not report any symptoms on the first day they started testing and they had a negative SARS-CoV-2 molecular and Ag-RDT tests. Our decision to pool performance across the different tests was also based on 1) findings not shown in this report that suggested the sensitivity of different Ag-RDT was similar to each other as a function of the viral load and 2) The study was not designed to evaluate differences in performance between the three different types of Ag-RDTs. Performance was calculated using sensitivity (rapid antigen positivity/comparator positivity) for single-day testing, two-times serial testing, and three-times serial testing for symptomatic and asymptomatic individuals based on day and patterns of positivity, as described in Table 1. Calculations for sensitivity were repeated with testing starting on different DPIPP. Current 95% confidence intervals are calculated using Clopper-Pearson method for binomial probability. To account for repeated measures within participants, we will update this preprint with confidence intervals estimated using bootstrapping technique.7 Negative Percent Agreement was calculated as a proportion of paired tests (Ag-RDT and molecular tests performed on the same-day) based on the following formula: True-Negative/[False-Positive+True-Negative] where True-Negative refers to instances a negative molecular test was paired with a negative Ag-RDT on the same day and False-Positive refers to a negative molecular test paired with a positive Ag-RDT test. All analyses were performed using R 4.1.1.8
Results
A total of 7,361 participants enrolled in the study, and 5,609 were eligible for this analysis. Of these, 154 individuals tested RT-PCR positive for SARS-CoV-2 during the study based on a composite definition described in Supplementary Table 1; 97 were without symptoms and 57 had symptoms at infection onset. Among the 5,455 participants who did not test positive, there were 39,682 days of paired Ag-RDT and RT-PCR testing, where comparator result was negative. Among these, 39,503 days had a concordant Ag-RDT negative result, yielding a negative percent agreement of 99.5% (95% CI: 99.5-99.6%).
Performance of Ag-RDT to detect SARS-CoV-2 on day of infection onset (DPIPP: 0) was higher among symptomatic participants (59.6%, 46.7-71.4%) in comparison to asymptomatic participants (9.3%, 5.0-16.7%) (Figure 2; Supplementary Table 2). Serial-testing with two Ag-RDT tests 48-hours apart (Symptomatic: 92.2%, 81.5-96.9%; Asymptomatic: 39.3%, 29.8-49.7%) and three Ag-RDT tests 48-hours apart (Symptomatic: 93.6%, 82.8-97.8%; Asymptomatic: 56.4%, 45.4-66.9%) improved performance of Ag-RDT. Notably, we observed that twenty participants had a singleton RT-PCR+, defined as a positive test preceded and followed by a negative RT-PCR test within 48-hours. Of those with singleton RT-PCR+ tests, none tested positive on Ag-RDT and only one had symptoms on the day of the singleton positive test. Excluding these participants did not impact sensitivity of Ag-RDT among symptomatic participants, but improved asymptomatic sensitivity to 11.7%, 50.7%, and 74.6% based on testing one, two, or three times with Ag-RDT at 48-hour intervals, respectively, on DPIPP 0.
a= participants replaced their assigned rapid antigen tests with commercially obtained rapid antigen tests; b= dates of RT-PCR testing could not be verified based on triangulation of self-reported, shipping, and resulting data
To approximate real-world scenarios, where a person may not necessarily start testing with Ag-RDT on the day of infection onset, we calculated performance separately on DPIPP 2, 4, 6, 8, and 10 (Figure 2; Supplementary Table 2) to approximate scenarios where a person started serially testing with Ag-RDTs on those days. Aggregated performance of Ag-RDT for DPIPP 0-6 among those who were symptomatic on a given DPIPP was 82.5% (76.8-87.0%) for single-timepoint testing but with a range of sensitivity from 59.6 to 94.8%. Serial-testing using two-time testing improved sensitivity to 93.4% (89.0-96.5%) and 94.3% (89.4-97.3%) for three-time testing. Sensitivity of performing single test, two-test serial testing, and three-test serial testing for asymptomatic people was 34.2% (27.4-41.7%), 55.6% (47.8-63.1%), 68.8% (60.5-76.2%), respectively during the first week of infection (DPIPP 0-6). Excluding singleton RT-PCR positive, the first-week (DPIPP 0-6) sensitivity was 38.5% (31.0-46.5%), 62.9% (54.7-70.6%), 79.2% (71.0-85.9%), respectively for testing one, two, or three times with Ag-RDT at 48-hour intervals.
Discussion
These findings from the largest study to date use paired Ag-RDT and RT-PCR testing for a comparative performance evaluation of Ag-RDTs among people with and without symptoms. These results provide compelling reasons to suggest that Ag-RDT occurrences and timeframes should be increased relative to the existing indications for use. These data suggest an improvement in test performance when symptomatic individuals test two times 48-hours apart using Ag-RDTs. Likewise, there are notable performance improvements in asymptomatic individuals when a single Ag-RDT test was followed by at least 2 subsequent tests at 48-hour intervals. Additionally, the rates of false positive results in the study were low; therefore, any Ag-RDT positive result should be considered positive without the need to retest. These results should be considered in the context of our study protocol which indicated testing at 48-hour intervals, and thus these data cannot support conclusions about serial testing for time intervals shorter than 48-hours.
These findings represent a comprehensive evaluation of the time dependent performance of Ag-RDT tests by studying the intended use population (i.e., symptomatic and asymptomatic individuals) throughout the course of molecular test positivity. Unlike previous reports, which used composite sampling methods and did not have sufficient longitudinal data to adequately evaluate performance of Ag-RDT from the onset of infection, we were able to approximate performance of Ag-RDT for symptomatic and asymptomatic users by comparing performance within the first week of infection to align with the indications listed in the EUA.9,10 The finding of singleton RT-PCR positive testing needs to be further investigated to understand the clinical significance of this observation.
Public health implications of our findings suggest that people who are suspected to be infected with SARS-CoV-2 should exercise caution despite an initial negative rapid antigen-test and favor mask-wearing and avoiding crowded places. In context of reports of viral culture positivity more than five days after initial positive test, our findings support isolation for a longer period of time to prevent the potential of spread of SARS-CoV-2 to others.11 Further research is needed to quantitatively estimate the benefits of Ag-RDT for early detection of infection and initiation of treatment, especially in settings where access to molecular testing is limited or molecular test results are delayed. Dissemination of clear guidance for appropriate testing using Ag-RDT based on data from this study may help preserve confidence in the performance of serial Ag-RDT to detect SARS-CoV-2 virus, especially as reports of individual false negative Ag-RDT from inadequate serial testing, contrary to the tests’ intended usage and guidance from the FDA, proliferate in lay media.
Data Availability
All data produced are available online at https://github.com/soni-lab.
Competing Interest Statement
DDM reports consulting and research grants from Bristol-Myers Squibb and Pfizer, consulting and research support from Fitbit, consulting and research support from Flexcon, research grant from Boehringer Ingelheim, consulting from Avania, non-financial research support from Apple Computer, consulting/other support from Heart Rhythm Society. YCM has received tests from Quanterix, Becton-Dickinson, Ceres, and Hologic for research-related purposes, consults for Abbott on subjects unrelated to SARS-CoV-2, and receives funding support to Johns Hopkins University from miDiagnostics.
Funding Statement
This study was funded by the NIH RADx Tech program under 3U54HL143541-02S2 and NIH CTSA grant UL1TR001453. The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the National Institute of Biomedical Imaging and Bioengineering; the National Heart, Lung, and Blood Institute; the National Institutes of Health, or the U.S. Department of Health and Human Services. Salary support from the National Institutes of Health U54HL143541, R01HL141434, R01HL137794, R61HL158541, R01HL137734, U01HL146382 (AS, DDM), U54EB007958-13 (YCM, MLR), AI272201400007C, UM1AI068613 (YCM), U54EB027049 and U54EB027049-02S1 (CJA, RLM).
Acknowledgment
We are grateful to our study participants and to our collaborators from the National Institute of Health (NIBIB and NHLBI) who provided scientific input into the design of this study and interpretation of our results, but could not formally join as co-authors due to institutional policies and to the Food and Drug Administration (Office of In Vitro Diagnostics and Radiological Health) for their involvement in the primary TUAH study. We received meaningful contributions from Drs. Bruce Tromberg, Jill Heemskerk, Dennis Buxton, Erin Iturriaga, Jue Chen, Andrew Weitz, and Krishna Juluru. We are thankful to county health departments across the country who helped with recruitment for this siteless study by spreading the word in their networks.
