Inflated false-negative rates in pooled RT-PCR tests of SARS-CoV-2 ================================================================== * Yair Daon * Amit Huppert * Uri Obolski ## Abstract **Background** Pooling is a popular strategy for increasing SARS-CoV-2 testing throughput. A common pooling scheme is Dorfman pooling: test *N* individuals simultaneously. If the first test is positive — retest each individual. **Methods** Using a probabilistic model, we analyze the false-negative rate (i.e., the probability of a negative result for an infected individual) of Dorfman pooling. Our model is conservative in that it ignores sample dilution effects, which can only worsen pooling performance. **Results** We show that one can expect a 60-80% increase in false-negative rates under Dorfman pooling, for reasonable parameter values. Moreover, we show that the false-negative rates under Dorfman pooling increase when the prevalence of infection decreases. **Discussion** In most pooling schemes, identifying an infected individual requires positive results in multiple tests and hence substantially deteriorates false-negative rates. Furthermore, this phenomenon is more pronounced when infection prevalence is low — exactly when pooling is most efficient. Thus, pooling presents an inherent trade-off: it is most efficient when it is least accurate. The deterioration of false-negative rates and the aforementioned trade-off are inherent problems of pooling schemes and should be kept in mind by practitioners and policy makers. ## 1 Introduction RT-PCR testing is a key component in breaking transmission chains and mitigating the COVID-19 pandemic. As such, the need for large-scale testing has resulted in development of pooling schemes of RT-PCR tests [1, 4, 5, 7, 8]. One such popular scheme is Dorfman pooling [1, 3]: Select *N* individuals and perform a single RT-PCR test on their combined (“pooled”) samples. If the pooled test yields a positive result — test each individual separately. The throughput efficiency of Dorfman pooling has been demonstrated empirically [1]. However, when test error rates are taken into consideration, a sharp increase in false-negative rates can be expected. It is important to distinguish three types of false-negative events when performing pooling. For convenience, we follow a single *infected* individual, henceforth referred to as “Donald”. A *single test’s* false-negative is the event of a negative result upon testing Donald separately, i.e., in an RT-PCR test without pooling. The probability of such an event is denoted *P*fn. A *pooled* false-negative occurs when a pooled test containing Donald’s sample (and other samples) yields a negative result, i.e., the pooling fails to detect at least one positive result. Lastly, a *scheme* false-negative results occurs when an entire pooling scheme fails to identify Donald as infected. Our goal is to calculate Dorfman’s scheme false-negative rate. Or: what is the probability of Dorfman pooling not identifying Donald as infected? ## 2 Methods ### 2.1 Probabilistic Assumptions We assume two pathways for a positive pooled test result: Viral RNA from an infected individual is correctly detected; or, some erroneous detection occurs (e.g. contaminant viral RNA is introduced). We ignore cross-reactivity with other Coronaviruses, which is negligible [11]. We assume a homogeneous and disconnected population (each individual is infected independently and with equal probability). For simplicity, we do not take into account sample dilution, since it can only further increase false-negative rates [1]. ### 2.2 Calculation of Dorfman’s scheme false-negative rate Denote the prevalence of infection in the (tested) population *q*. As before, *P*fn denotes the single RT-PCR test’s false-negative rate. We also denote *P*fp the probability of an erroneous RNA detection, which may cause a false-positive. By our assumptions, a pool containing Donald’s sample and *N −* 1 other samples will yield a *negative* result if all of the following occur: * No erroneous detection occurred (i.e. no false-positive). This happens with probability 1 *− P*fp. * The detection process fails for Donald’s sample. A false-negative occurs, with probability *P*fn. * No detection for any of the other *N −* 1 samples. For a single sample, the probability of being detected is the prevalence of SARS-CoV-2 in the tested population *q*, multiplied by the true-positive rate. But the true-positive rate is the complement of *P*fn, namely 1 *− P*fn, hence the probability of detection is *q*(1 *− P*fn). For *N −* 1 such samples, the probability of *not* being identified is (1 *− q*(1 *− P*fn))*N−*1. The pooled false-negative probability for Donald is simply the product of the terms above. Hence: ![Formula][1] If the pooled test yields a positive result, Donald is tested separately. We assume such a simple procedure poses no risk of introducing contaminant RNA. Therefore, the separate test yields a positive result with probability 1 *− P*fn. We calculate the probability that Donald is mistakenly identified as not infected — the scheme’s false-negative rate — denoted *P*sfn below. To *correctly* identify Donald as infected, both pooled and separate tests have to yield a positive result. Thus, the scheme’s false-negative rate *P*sfn is the complement of the product of the two previous terms: ![Formula][2] ### 2.3 Comparison metric The single test false-negative *P*fn and scheme false-negative rate *P*sfn are compared via: ![Formula][3] *E*rel is the percentage increase in the pooling scheme false-negative rate, relative to the single test false-negative rate. ## 3 Results We plot *E*rel for varying prevalence *q* and false-negative *P*fn values. As recommended by [1], we apply different pool sizes *N*, for different prevalence values. We observe that for a false-positive rate *P*fp = 0.05 [1] and a range of reasonable false-negative and prevalence values [10–13], an increase of at least 60% in *E*rel can be expected (Figure 1). Interestingly, an increase in infection prevalence monotonically decreases the scheme false-negative rate, as can also be easily seen from equation (2). For the chosen parameter ranges, the increase in the single test false-negative rates increases the relative error *E*rel. These effects can be seen in Figure 1, upon conditioning on pool size. Finally, extending the range for *P*fp yields no qualitative differences. ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/12/08/2020.12.02.20242651/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2020/12/08/2020.12.02.20242651/F1) Figure 1: Relative increase in Dorfman pooling false-negative rates *E*rel. Color represents the relative percentage increase in the scheme false-negative rates relative to the single test false-negative rates, *P*fn. The disease prevalence, *q*, is varied on the x-axis, while the single test false-negative rate is varied on the y-axis. Note that Pool size, *N*, was chosen according to *q* as in [1]. ## 4 Discussion Although Dorfman pooling improves testing throughput, we have shown that it can increase *E*rel — the false-negative rates relative to individual testing. Furthermore, low values of infection prevalence, or low values of single test false-negative rates, increase *E*rel. These results remain qualitatively similar under varying parameter values, in the observed ranges [6, 10–12] (Figure 1). Our results lead to a fact almost disregarded in [1]: although (Dorfman) pooling is most efficient when prevalence is low, such circumstances are exactly those leading to a substantial increase in false-negative rates. An approximation to our results has been previously considered: *P*sfn *≈* 1 *−* (1 *− P*fn)2 [2, 9]. Such an approximation arises when one assumes that the probability of a false-negative is identical for the pooled and single tests. Although simplistic in nature, this approximation does capture the intuition behind our results: For Donald to be considered negative under Dorfman pooling, he has to test negative twice. In addition to this approximation only being an upper bound [2], it does not account for the effect of infection prevalence on the false-negative rates under Dorfman pooling. Supporting the importance of this effect, such an association between prevalence and the scheme false-negative rate under Dorfman pooling has been empirically noted [1]. Although we have shown the inherent risk of Dorfman pooling, this short-coming applies to other pooling schemes. Pooling schemes (e.g. [4, 14]), require some sequence of positive pooled results to correctly identify Donald as infected. Consider the pooling scheme of [14]: If the first pool yields a positive result, it is split in two. Then the splitting is repeated until resulting pools are negative or individuals are tested separately. With an initial pool size of 32, Donald will necessarily have to test positive in pools of size 32, 16, 8, 4 and 2, as well as in a single test, for the scheme to correctly identify him as infected. Compare this to the Dorfman scheme that requires a positive test in a pool of size *N* = 8, and an additional single positive test to identify Donald as infected. The pooling scheme of [14] will necessarily yield more false-negatives than Dorfman pooling — there are additional places for it to fail. As mentioned in [1], introducing a positive dependence within a pool decreases the false-positive rate. In the extreme case, consider a fully connected pool, where one infection implies the entire pool is infected. In this case, a calculation analogous to the one conducted above recovers the initial false-negative rate *P*fn. Interestingly, pooling was also noted to have increased throughput when infection probabilities are dependent between the pooled individuals [1], providing another advantage to sampling dependent individuals in pooling schemes. To conclude, pooling is an important technique which can improve testing throughput in a cost-effective manner. Nevertheless, a substantial deterioration in pooling schemes’ false-negative rates can be expected. Such a deterioration has crucial implications for controlling the spread of COVID-19. ## Data Availability No data was used in the manuscript. ## Footnotes * Modified manuscript to refer to some papers and emphasize the trade-off between false-negative and efficiency. * Received December 2, 2020. * Revision received December 8, 2020. * Accepted December 8, 2020. * © 2020, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. [1]. Netta Barak, Roni Ben-Ami, Tal Sido, Amir Perri, Aviad Shtoyer, Mila Rivkin, Tamar Licht, Ayelet Peretz, Judith Magenheim, Irit Fogel, et al., Lessons from applied large-scale pooling of 133,816 SARS-CoV-2 RT-PCR tests, MedRxiv (2020). 2. [2]. Alhaji Cherif, Nadja Grobe, Xiaoling Wang, and Peter Kotanko, Simulation of pool testing to identify patients with coronavirus disease 2019 under conditions of limited test availability, JAMA network open 3 (2020), no. 6, e2013075–e2013075. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2020.13075&link_type=DOI) 3. [3]. Robert Dorfman, The detection of defective members of large populations, The Annals of Mathematical Statistics 14 (1943), no. 4, 436–440. 4. [4]. US Food, Drug Administration, et al., Accelerated emergency use authorization (eua) summary COVID-19 RT-PCR test (laboratory corporation of america). 5. [5]. Rudolf Hanel and Stefan Thurner, Boosting test-efficiency by pooled testing strategies for SARS-CoV-2, arXiv preprint arxiv:2003.09944 (2020). 6. [6]. Lauren M Kucirka, Stephen A Lauer, Oliver Laeyendecker, Denali Boon, and Justin Lessler, Variation in false-negative rate of reverse transcriptase polymerase chain reaction—based SARS-CoV-2 tests by time since exposure, Annals of Internal Medicine (2020). 7. [7]. Stefan Lohse, Thorsten Pfuhl, Barbara Berkó-Göttel, Jürgen Rissland, Tobias Geißler, Barbara Gärtner, Sören L Becker, Sophie Schneitler, and Sigrun Smola, Pooling of samples for testing for SARS-CoV-2 in asymptomatic people, The Lancet Infectious Diseases (2020). 8. [8].Rodrigo Noriega and Matthew Samore, Increasing testing throughput and case detection with a pooled-sample Bayesian approach in the context of COVID-19, bioRxiv (2020). 9. [9]. Alexander Pikovski and Kajetan Bentele, Pooling of coronavirus tests under unknown prevalence, Epidemiology & Infection 148 (2020). 10. [10]. Jessica Watson, Penny F Whiting, and John E Brush, Interpreting a COVID-19 test result, BMJ 369 (2020). 11. [11]. L Wijsman, R Molenkamp, CBEM Reusken, A Meijer, et al., Comparison of seven commercial RT-PCR diagnostic kits for COVID-19., Journal of clinical virology: the official publication of the Pan American Society for Clinical Virology 128 (2020), 104412–104412. 12. [12]. Paul Wikramaratna, Robert S Paton, Mahan Ghafari, and Jose Lourenco, Estimating false-negative detection rate of SARS-CoV-2 by RT-PCR, medRxiv (2020). 13. [13]. Steven Woloshin, Neeraj Patel, and Aaron S Kesselheim, False negative tests for SARS-CoV-2 infection — challenges and implications, New England Journal of Medicine (2020). 14. [14]. Idan Yelin, Noga Aharony, Einat Shaer-Tamar, Amir Argoetti, Esther Messer, Dina Berenbaum, Einat Shafran, Areen Kuzli, Nagam Gandali, Tamar Hashimshony, et al., Evaluation of COVID-19 RT-qPCR test in multi-sample pools, MedRxiv (2020). [1]: /embed/graphic-1.gif [2]: /embed/graphic-2.gif [3]: /embed/graphic-3.gif