The impact of high frequency rapid viral antigen screening on COVID-19 spread and outcomes: a validation and modeling study

Direct Antigen Rapid Test Accuracy Correlates with Viral Load Levels

Rapid antigen tests have recently been considered a viable source for first-line screening, although concerns about the accuracy of these tests persist. We developed a direct antigen rapid test in a lateral flow dipstick format for the detection of the spike glycoprotein from SARS-CoV-2 in nasopharyngeal swab specimens. Of the total number of nasopharyngeal swab specimens evaluated by qRT-PCR for amplification of SARS-CoV-2 RNA-dependent RNA polymerase (RdRp), nucleocapsid (N), and envelope (E) genes, 72 tested positive and 49 tested negative (Table 1).

The overall sensitivity and specificity of the rapid antigen test was 84.7% and 85.7%, respectively (Table 1). Our data demonstrate that the sensitivity of our test is positively correlated to the viral load level (Fig. 1, Table 2). The rapid test result was compared to the qRT-PCR cycle threshold (Ct) value measured across RdRp, N, and E genes, and calculated as sensitivity and specificity.

• Download figure
• Open in new tab

Figure 1.

Performance of direct antigen rapid test (DART) for the detection of SARS-CoV-2 in 121 retrospectively collected patient nasopharyngeal swab specimens. Percentile Positive ranks the samples in order of high Cycle threshold (Ct) value to low Ct value. DART sensitivity is determined by calculating true positive agreement to qRT-PCR; the plot uses an ax^b+c fit and 95% confidence intervals for the sensitivity. Shown are the percentile positive cases of the total positive population conditioned to qRT-PCR Ct for RdRp (RNA-dependent RNA polymerase), N (nucleocapsid), and E (envelope) genes, and the average of these genes.

Ct value cutoffs represent the number of qRT-PCR cycles at which generated fluorescence crosses a threshold during the linear amplification phase. The x-axis of Figure 1 describes the percentile positives in the total positive population. Percentile Positives ranks the samples in order of high qRT-PCR Ct value to low. In other words, the higher the percentile, the more “positive” because fewer qRT-PCR cycles are required for gene detection. Because the Ct value is a variable unit based upon qRT-PCR protocol and instrumentation, we evaluated sensitivity against the percentile of cases across Ct values.

As the percentile positive increases, the sensitivity between the rapid test and the gold-standard qRT-PCR increases, reaching 100% sensitivity at 68.1% percentile positive for the gene average detection. Significantly, even at percentile positives between 0% and 68.1%, the sensitivity remains above 80%. Taken together, the clinical data shows that the rapid antigen test performs with increasing accuracy for individuals with a higher viral load, and are thus the most infectious^13-15.

An Enhanced Epidemiological SIDHRE-Q Model

We propose an enhanced epidemiological modeling system, SIDHRE-Q, a variant of the classical SIR model in order to expand our clinical validation study and to understand the effects of using frequent rapid tests such as the rapid antigen test on COVID-19 outbreak dynamics. The changes we make to the basic model to encompass the unique characteristics of the COVID-19 pandemic are similar to those presented by Giordano et al.¹⁶ (Fig. 2). The differential equations governing the evolution of the SIDHRE-Q model and descriptions of the parameter values are provided in the material and methods section (Equation 2, Table 5).

• Download figure
• Open in new tab

Figure 2.

Graphical scheme displaying the relationships between the stages of quarantine and infection in SIDHRE-Q model: Q-U, quarantine uninfected; S, susceptible (uninfected); I, infected undetected (pre-testing and infected); D, infected detected (infection diagnosis through testing); H, hospitalized (infected with life threatening symptom progression); R, recovered (healed); E, extinct (dead); and Q-R, quarantine recovered (healed but in quarantine by false positive testing).

Figure Supplement 1. Graphical scheme displaying parameters between the stages of quarantine and infection in SIDHRE-Q model.

An individual that begins in S may either transition to a Quarantine Uninfected (QU) state via a false positive result or to an Infected Undetected (I) state via interaction with an infected individual. Should an individual in S move into Q-U, they are quarantined for 14 days before returning to S, a time period chosen based on current knowledge of the infectious period of the disease. One could also conceive of an effective strategy in which individuals exit quarantine after producing a certain number of negative rapid tests in the days following their initial positive result or confirm their negative result using qRT-PCR.Given that those diagnosed are predominantly quarantined, individuals in I interact more with the S population than do those in Infected Detected (D). Therefore, the infectious rate for I is assumed to be significantly larger than for D. Furthermore, a region’s ability to control an outbreak is directly related to how quickly and effectively people in I test into D, reducing their infectiousness through quarantine. This study, in particular, highlights the critical role frequency of testing, along with strict quarantine, has in mitigating the spread of the disease and provides specific testing strategies based on rapid tests we predict to be highly effective.

In this model, we assume that individuals receive a positive diagnosis before developing severe symptoms and that those with symptoms severe enough to be potentially fatal will go to the hospital. If an individual develops symptoms, we assume they are tested daily until receiving a positive result; hence, before severe symptoms develop, they will be diagnosed with high probability. Those who do not develop symptoms are tested according to the frequency of tests administered to the general population. Therefore, there is no modeled connection between I and H or between I and E. Removing these assumptions would have negligible impact on the results as these flows are very small.

Should an individual test positive and transition to D, they may either develop serious symptoms requiring care or recover. Those who develop serious symptoms and transition to state H will then transition to either R or E. The recovered population is inevitably tested, as infected individuals may recover without being detected. Therefore, the Quarantined Recovered (Q-R) state is introduced with the same connections to R as the connections between S and Q-U. Though the reinfection rate of SARS-CoV-2 has been a point of recent debate, it is assumed that the number of re-infected individuals is small^27-31. Therefore, individuals cannot transition from R to S, hence the separately categorized quarantined populations.

We considered several variations and extensions of the SIDHRE-Q model. In simulations, we tested additional states, such as those in the SIDARTHE model, which include distinctions between symptomatic and asymptomatic cases for both detected and undetected populations¹⁶. Incorporating information about the correlations between viral load and infectivity and sensitivity were also considered. Altogether, our modeling system has been well tuned to predict the impact of high frequency rapid testing on COVID-19 spread and outcomes.

Frequent Rapid Testing with Actionable Quarantining Dramatically Reduces Disease Spread

In order to demonstrate how strategies could affect the disease spread in different geographies and demographics, we used surveillance data obtained from regions of varying characteristics: the state of Massachusetts (MA), New York City (NYC), Los Angeles (LA), and São José do Rio Preto (SJRP), Brazil, the site of the rapid antigen test clinical validation study. These regions are also selected in our study due to the readily available surveillance data provided by the local governments. We fit the model to the data from each region starting 1 April 2020. At this time point the disease reportedly is most advanced in NYC and least advanced in SJRP, Brazil with estimated cumulative infection rates of 7.11% and 0.12%, respectively.

After calibrating the SIDHRE-Q model, the disease spread is observed with varying validated rapid antigen test performances and frequencies (Fig. 3). Sensitivity (the ratio of true positives to the total number of positives) and specificity (the ratio of true negatives to the total number of negatives) compared to gold-standard qRT-PCR were used as measures of test accuracy.

• Download figure
• Open in new tab

Figure 3.

COVID-19 Outcomes in 3 US Regions and Brazil as a result of Frequent Rapid Testing Protocol using the SIDHRE-Q Model. The Cumulative Detected Infected, Hospitalized, Deceased, Active Infections, Recovered, and Quarantined are modeled over 105 days (top to bottom) using reported data from 4 global regions: Massachusetts, Los Angeles, New York City, and São José do Rio Preto in Brazil (left to right). The COVID-19 population spread and outcomes are modeled under a Rapid Testing Protocol (sensitivity 80%, specificity 90%) with variable testing frequencies ranging from 1-21 days between tests. This protocol is compared to a symptom-based Rapid Testing protocol and a symptom-based qRT-PCR protocol.

Figure Supplement 1. COVID-19 Outcomes as a result of Frequent Rapid Testing Protocol with variable test performances using SIDHRE-Q Model.

Figure Supplement 2. Time series of the four fitted parameters α, ν, μ, and τ.

The rapid test frequency is varied while maintaining an accuracy of 80% sensitivity and 90% specificity, comparable to our clinical data collected in SJRP, Brazil. These testing scenarios are then compared to symptomatic testing, in which individuals receive a rapid test only when presenting symptoms, via either a rapid test or qRT-PCR. Since the primary testing regiment deployed in MA, LA, NYC and SJRP, Brazil is qRT-PCR-based and focused on symptomatic individuals, the symptomatic testing protocol via qRT-PCR is directly estimated from the data to be the rate v (Table 5).

The difference between the qRT-PCR and rapid test simulations (red and orange lines, respectively) is therefore only sensitivity of testing (Fig. 3). We assumed that test outcome probability is a function only of whether an individual is infected and independent of other factors; one can consider this a lower bound on effectiveness of a strategy, as sensitivity and infectivity are often positively correlated with antigen testing.

To better understand the effect of rapid testing frequency and performance on healthcare capacity and mortality rates, we simulate the testing strategy with 30%-90% sensitivity each with 80% or 90% specificity against the symptomatic testing strategy. Table 3 describes the different test performances implemented in the model matched with corresponding Figure 3 – Figure Supplement 1 plots.

As per our hypothesis, frequency and symptom-based testing dramatically reduced infections, simultaneous hospitalizations, and total deaths when compared to the purely symptom-based testing regiments, and infections, hospitalization, and death were reduced as frequency increased. Although testing every day was clearly most effective, even testing every fourteen days with an imperfect test gave an improvement over symptomatic testing with qRT-PCR. While the strategy works best when implemented at the very beginning of an outbreak, as demonstrated by the results in SJRP, Brazil, it also works to curb an outbreak that is already large, as demonstrated by the results in NYC. The difference between frequencies is more noticeable when the testing strategy is applied to the outbreak in NYC, leading us to hypothesize that smaller outbreaks require a lower testing frequency than larger ones; note the difference between the dependence on frequency to curb a small initial outbreak in SJRP, Brazil versus a large one in NYC (Fig. 4, Figure 4 – Figure Supplement 1).

For test performance of 80% sensitivity and 90% specificity, the percent of the population that has been infected in total from the beginning of the outbreak to mid-July drops from 18% (MA), 11% (LA), 26% (NYC), and 11% (SJRP, Brazil) to 3%, 2%, 12%, and 0.26%, respectively, using a weekly rapid testing and quarantine strategy (with regards to predictions of overall infection rates, other studies based on seroprevalence and epidemiological predictions have reached similar conclusions^32,33). If testing is increased to once every three days, these numbers drop further to 1.6% (MA), 1.4% (LA), 9.4% (NYC), and 0.19% (SJRP, Brazil) (Table 4).

To further examine the relationship between frequency and sensitivity, we modeled the maximum number of individuals in a given state over the 105-day time period for four geographic regions (Fig. 4). In all four geographic regions, as frequency of testing increases, the total infections, maximum simultaneous hospitalizations, and total deaths converge to small percentages regardless of the sensitivity at high frequencies. It is clear that the difference in frequency required to achieve the same result using tests of differing sensitivities is very small (Fig. 4) For example, we predict that for the outbreak in LA, a testing strategy started on 1 April of every 10 days using a test of sensitivity 90% would have resulted in 2.5% of the population having been infected, while using a test of sensitivity 30% would require a strategy of every 5 days to achieve the same number. Thus, we conclude that frequency is more important than sensitivity in curbing the spread, and a large range of sensitivities prove effective when testing sufficiently often. How frequently, exactly, depends on the specific outbreak and what stage it is in, which leads us to the location-based deployment strategy discussed in a later section. Frequency of testing can be significantly reduced to effectively contain the disease once the initial outbreak has been controlled; it is clear that this takes only a matter of weeks (Fig. 3).

• Download figure
• Open in new tab

Figure 4.

Effect of Rapid Testing Protocol under variable testing sensitivities and increasing frequency under the SIDHRE-Q Model. The Cumulative Infections, Maximum Simultaneously Hospitalized, and Deceased populations are modeled for Massachusetts, Los Angeles, New York City, and São José do Rio Preto in Brazil. The effect of increasing frequency of testing is modeled for various testing sensitivities (30%-90%) with a 90% specificity.

Figure 4 – Figure Supplement 1. Effect of Rapid Testing Protocol under variable testing sensitivities and increasing frequency under the SIDHRE-Q Model (80% specificity).

On the other hand, according to the specificity of the rapid test and the quarantine duration, larger testing frequency result in a larger percent of the population quarantined (Fig. 3). Assuming a 90% rapid test specificity and 14-day quarantine duration, for the 1, 3- and 7-day frequencies almost 60%, 38% and 20% of the population, respectively, would be quarantined. This figure may be reduced with additional rules for exiting quarantine early, such as after complementary testing. An example of such a strategy is that individuals who test positive are required to either quarantine for two weeks or produce two consecutive negative rapid tests in the two days following their positive result. Assuming 80% sensitivity and 90% specificity, those individuals will reenter the public while still infected with probability 0.04. If uninfected, that individual will exit quarantine after two days with probability 0.81. However, a compromise between the reduction of infections and the proportion of the population in quarantine would be part of the planning for the appropriate testing protocol in each community or region.

Additionally, while high frequency may be necessary to contain a large outbreak initially, relatively infrequent testing, such as every one or two weeks, is sufficient to keep controlled outbreaks small, while reducing the number of quarantined individuals to less than 10% of the population using a two-week mandatory quarantine.

A County-Based Testing Strategy Offers a Cost-effective Approach to Large-scale COVID-19 Surveillance

To examine the effects of resource-strategic testing schemes, we modeled the COVID-19 prevalence by varying testing frequency across counties of California. For this analysis, only California was analyzed because of the accessibility of the county level data. In this scheme, the percent of active infected detected individuals in a county determines the frequency of testing. We define thresholds for the number of active detected infections that, when hit, initiate testing protocols of different frequencies depending on the threshold hit. We first tested evenly spaced thresholds for the number of detected active infections up to 1% of the population, but later adopted thresholds that were determined according to Equation 1. In Equation 1, D = population of state D at the time of testing. T = number of active infections which, if reached, initiates everyday testing. The days between tests are rounded to the closest integer value.

The days between tests are chosen such that the detected active infections should remain near to or below T. If the initial detected active infections are greater than T, then the testing frequency of 1 will cause infections to rapidly drop. Both the threshold at which everyday testing begins and the coefficient of log₂T/D can be modified to produce a strategy that is more or less frequent in testing or resource effective; a range of days between tests from 14 days to 1 day are used (Fig. 5). A scan over different choices of T is shown in the supplements to Figure 5; the threshold we choose in Figure 5 is 0.05% because it is successful in curbing the outbreak within the time period we consider. Whilethese choices work for the epidemic in California at the point we start our simulations, 10 April, they do not necessarily reflect the most resource effective choices everywhere. Our analysis could be redone to select the best strategy in other states or in the country as a whole.

• Download figure
• Open in new tab

Figure 5.

Effect of County Based Rapid Test Protocol (A) and Uniform qRT-PCR Protocol (B) on active infected detected population over time in California (CA). The legend denotes the thresholds at which testing frequency is determined, the testing frequencies, the percent of CA population under the strategy, and the cost per person per day.

Figure Supplement 1. Effect of County Based Rapid Testing strategy on COVID-19 outcomes in California.

Figure Supplement 2. Time series of the three fitted pieces of data Cumulative Cases, Daily Hospitalized, and Cumulative Deaths for each CA county receiving testing.

Using a rapid test with a sensitivity of 80% and specificity of 90%, the county-based testing with threshold 0.05% reduces the active infections from 0.94% to 0.0005%, while the uniform strategy with tests administered every 7 days results in double the number of active infections (Fig. 5). As the threshold is reduced, the total cost increases while thecumulative infections, maximum percentage hospitalized, and cumulative deaths all decrease (Figure 5 – Figure Supplement 1).

Strategy B in Figure 5 consists of qRT-PCR testing uniformly applied to the highlighted population with a frequency of once weekly. The average cost per person per day is just under $15. Despite this frequency and the accuracy of qRT-PCR, the strategy does not succeed in curbing the spread as fast as strategy A, which uses a testing sensitivity and specificity of 80% and 90%, respectively, and testing frequency that vary between counties depending on the proportion of their population that is currently infected. The total cost for strategy A is estimated at a fraction of the other at $1.53 per person per day.

Development of Direct Antigen Rapid Test for the Detection of SARS-CoV-2

We developed a direct antigen rapid test for the detection of the spike glycoprotein from SARS-CoV-2 in nasopharyngeal swab specimens as previously described⁴¹. Briefly, the rapid antigen test is an immunochromatographic format with a visual readout using anti-spike mouse monoclonal antibodies (E25Bio, Inc., Cambridge, MA, USA) that are either coupled to 40 nm gold nanoparticles (Abcam, Cambridge, UK) or adsorbed to nitrocellulose membranes (Sartorius, Goettingen, Germany). Each rapid antigen test has a control area adjacent to the paper absorbent pad; the control is an anti-mouse Fc domain antibody (Leinco Technologies, Fenton, MO, USA) that will capture any of the antibody-conjugated gold nanoparticles to generate a control visual signal. A visual signal at the test area reflects SARS-CoV-2 spike glycoprotein that is “sandwiched” between an anti-spike glycoprotein antibody adsorbed to the nitrocellulose membrane and a second anti-spike glycoprotein antibody covalently coupled to visible gold nanoparticles.

Validation of Direct Antigen Rapid Test for the Detection of SARS-CoV-2

In a retrospective study of nasopharyngeal swab specimens from human patients, we compared the accuracy of the rapid antigen test to the viral load of individuals. Nasopharyngeal swab specimens (n = 121) were tested in Brazil following approved human subjects use protocols. The age of study participants ranged from 1 to 95 years with an overall median of 37 years (interquartile range, 27-51 years), and 62% were female. The demographic summary of the patients are included in Supplementary Table 1. The nasopharyngeal swab specimens were banked refrigerated or frozen samples from suspected patients submitted to the lab for routine COVID diagnosis. Prior to using the rapid test, the nasopharyngeal swab samples were validated by qRT-PCR using GeneFinder™ COVID-19 Plus RealAmp Kit (OSANGHealtcare, Anyang-si, Gyeonggi-do, Republic of Korea I). The primary study under which the samples and data were collected received ethical clearance from the Faculdade de Medicina de São José do Rio Preto (FAMERP), protocol number 31588920.0.0000.5415. All excess samples and corresponding data were banked and de-identified prior to the analyses.

A nasopharyngeal swab specimen (1 mL) was concentrated using a Vivaspin 500 centrifugal concentrator (Sartorius, Goettingen, Germany) at 12,000 x g for 10 minutes. The concentrated nasopharyngeal swab specimen retentate was transferred to a collection tube and the rapid antigen test was inserted into the tube with the retentate and allowed to react for 15 minutes. After processing of the rapid antigen test, the visual positive or negative signal was documented.

Data for Modeling

As of August 2020, the United States and Brazil have the highest number of confirmed COVID-19 cases and deaths worldwide, with both countries reporting their first case on 26 February 2020¹. Although several affected US regions could have been modeled, we look at data from Massachusetts, New York, and Los Angeles: these regions each contained “hotspots”, or areas of surging COVID-19 cases, at different points in time during the pandemic and have publicly available government-provided surveillance data. Our model is fit using data over 105 days beginning on April 1 for Figures 3 and 4 and 105 days beginning on April 10 for Figure 5 (see “Modeling Parameters” in Methods). In order to understand the various testing proposals on a global scale, we performed our clinical study in and expanded the modeling study to Brazil. The specific data we use to fit our model are cumulative confirmed cases, total deaths, and number of daily hospitalizations due to COVID-19. This surveillance data was retrieved from government-provided online databases^42-48.

Modeling Parameters

Equation 2 below provides the exact differential equations governing the model.

In order to determine the values of the parameters defining the flows between states, we use a least squares regression performed at seven day intervals in the datasets to which we fit. This allows the model to take into account the time dependent nature of the parameters, which rely on factors such as social distancing regulations and changes in testing capacity. We also fit window sizes between 1 and 21 days and find that while the fit degrades with larger window size, the overall shape of the fits do not change. We choose seven days assuming policy changes take a week to become effective and that reasonable parameters can be expected to change within this time period. Also, the seven day window size accounts for the fact that often data is not reported as diligently over the weekend. Time series of the values of the parameters for the geographic locations discussed in this paper can be found in the supplemental materials for Figure 2.

Given the restrictions on data available for the populations of various states, varying all of the parameters results in an over parameterized system. Therefore, a subset of the model parameters are fit while the others are either extracted from other sources; see Table 5.

The fitting procedure minimizes the sum of the squared residuals of the total cases, current daily hospitalizations, cumulative deaths, and percentage of total infected individuals currently hospitalized. The first three are present in the data sets while the latter is derived from the estimates of the ratio between infected undetected to infected detected individuals from the CDC Laboratory Seroprevalence Survey Data⁴⁹. While this ratio changes over time, the percentage of infected individuals developing severe symptoms should remain roughly constant throughout the course of the epidemic in the different locations studied.

We consider the data sets for outbreaks in MA, NYC, LA, and SJRP, Brazil^42-47. While each location has testing and fatality information dating back to January, hospitalization data was not included until late March (for NYC and SJRP) and April (for MA and LA). Hence we begin our fitting procedure and testing strategy on 1 April for each of the data sets; by this point, the outbreak is advanced in NYC, substantial in MA, non-negligible, but far from its peak, in LA, and in early stages in SJRP, Brazil. Starting simulations at various stages of the outbreak allows one to see the difference in results between when a testing strategy is administered.

In order to determine the effectiveness of the county-based strategy when applied to the state of California, we also fit all of the counties in California with a population greater than 1.5% of that of the entire state and with greater than zero deaths. The results do not depend on these selections, but instead suggest a practical criteria to administer limited resources. The fitting is done starting 10 April for these counties, as at this point the outbreak is sufficiently well-documented in each to successfully model. For the county-level data we compute a seven day running average of each of the data sets to which we then fit in order to smooth out fluctuations in the data, likely due to reporting, which are more significant here than in the other data sets considered, as the county populations are smaller and hence discrepancies impact the smoothness of the data more. The fits for each of the counties can be found in the supplementary materials to Figure 5.

As one can see from Figure 2, these data sets are particularly not smooth, which indicates inefficiencies in reporting. Additionally, it is difficult to gauge their consistency within the dates provided or to compare between locations, as reporting mechanisms changed over time within the same locations. Despite this lack of consistency, our model and fitting mechanism was successful in reproducing the progress of the outbreak in each data set studied.

The authors confirm that the data supporting the findings of this study are available within the article and/or its supplementary materials; any other data will be made available upon request. Our code can be found on github: https://github.com/badeaa3/COVID19 Rapid Testing. The code is written using python with the packages scipy, numpy, lmfit, matplotlib and plotly^50-54.