Diagnostic accuracy of RT-PCR for detection of SARS-CoV-2 compared to a “composite reference standard” in hospitalized patients
===================================================================================================================================

* Noah Reich
* Christopher F. Lowe
* David Puddicombe
* Nancy Matic
* Jesse Greiner
* Janet Simons
* Victor Leung
* Terry Chu
* Hiten Naik
* Nick Myles
* Laura Burns
* Marc G. Romney
* Gordon Ritchie
* Sylvie Champagne
* Kent Dooley
* Inna Sekirov
* Aleksandra Stefanovic

## Abstract

**Background** COVID-19 caused by the novel coronavirus SARS-CoV-2 has caused the greatest public health emergency of our time. Accurate laboratory detection of the virus is critical in order to contain the spread. Although real-time polymerase chain reaction (PCR) has been the cornerstone of laboratory diagnosis, there have been conflicting reports on the diagnostic accuracy of this method.

**Methods** A retrospective review was performed on all hospitalized patients tested for SARS-CoV-2 (at St. Pauls Hospital in Vancouver, BC) from March 13 – April 12, 2020. Diagnostic accuracy of initial PCR on nasopharyngeal (NP) swabs was determined against a composite reference standard which included a clinical assessment of the likelihood of COVID-19 by medical experts, initial and repeat PCR, and post-hoc serological testing.

**Results** A total of 323 patients were included in the study, 33 (10.2%) tested positive and 290 (89.8%) tested negative by initial PCR. Patients testing positive were more likely to exhibit features of cough (66.7% vs 39.3%), shortness of breath (63.6% vs 35.9%), fever (72.7% vs 27.6%), radiographic findings (83.3% vs 39.6%) and severe outcomes including ICU admission (24.2% vs 9.7%) and mortality (21.2% vs 6.2%) compared to patients testing negative. Serology was performed on 90 patients and correlation between serology and PCR was 98.9%. There were 90 patients included in the composite reference standard. Compared to the composite reference standard, initial PCR had sensitivity of 94.7% (95% CI 74.0 to 99.9%), specificity of 100% (95% CI 94.9 to 100%), positive predictive value of 100% (95% CI 81.5 to 100%) and a negative predictive value of 98.6% (95% CI 92.5 to 100%).

**Discussion** Our study showed high sensitivity of PCR on NP swab specimens when compared to composite reference standard in hospitalized patients. High correlation of PCR with serological testing further increased confidence in the diagnostic reliability of properly collected NP swabs.

Keywords
*   Coronavirus
*   COVID-19
*   SARS-CoV-2
*   diagnostic accuracy
*   sensitivity
*   serology

## Introduction

COVID-19, a disease caused by the novel coronavirus virus SARS-CoV-2, has caused an unprecedented global pandemic and rapidly developed into the greatest public health emergency of our time [1] [2].The number of COVID-19 cases has reached over 100,000,000 and resulted in more than 2,000,000 deaths globally at the time of this manuscript [3]. Accurate laboratory detection has been paramount to controlling SARS-CoV-2 spread through identification of infected individuals and areas of outbreak in the community and healthcare facilities. Furthermore, expeditious testing of infected individuals has been critical to the public health response of rapid case identification, contact tracing and quarantine [4].

Clinical manifestations of COVID-19 range from asymptomatic or mild infections to severe pneumonia, respiratory failure and death [5]. While most cases of COVID-19 are mild, up to 20% require hospitalization and 5% intensive care [6] [7]. Hospitalized patients most commonly present with fever, cough, shortness of breath, myalgia, fatigue, lymphopenia and bilateral ground glass opacities on chest imaging. [6] [7] [8]

Molecular methods such as PCR are the mainstay for the diagnosis of COVID-19 in hospitalized patients. Significant differences in molecular assays exist with respect to extraction methods, target detection, specimen validation, instrumentation, and analytical sensitivity. The SARS-CoV-2 genome has been well characterized allowing for primer development targeting different components of the viral genome [9]. Commonly used primer targets include RdRP gene (RNA-dependent RNA polymerase), E gene (envelope glycoprotein), N gene (nucleocapsid phosphoprotein), ORF 1a/b genes (open reading frame), and S gene (spike glycoprotein). As part of the *in vitro* diagnostic validation and regulatory approval, molecular assays undergo assessment of analytic sensitivity and specificity; however, clinical diagnostic performance is less well described. Reports of initial false negative SARS-CoV-2 results by PCR, later diagnosed with COVID-19 by chest CT and repeat PCR, have emerged [10] [11]. As SARS-CoV-2 serological testing has become available, this has presented another testing modality to retrospectively adjudicate suspected cases with negative PCR [12]. The sensitivity and specificity of serology are estimated in the range of 90-100% and 95-100%, respectively [13].

As no single reference standard for diagnosing COVID-19 exists, we aim to determine the clinical sensitivity, specificity, positive and negative predictive values of a SARS-CoV-2 PCR assay in acutely ill patients admitted to hospital using a composite reference method including clinical assessment, molecular testing and serology for SARS-CoV-2.

## Methods

### Participant selection

All adult patients aged ≥18 years admitted to an acute care hospital for >24 hrs tested by PCR for SARS-CoV-2 from March 13 to April 17, 2020 were included in the study. Patients were excluded from the study if they had testing performed >40 days after symptom onset [14].

### Data collection

Data were collected retrospectively from the hospital’s electronic medical record system. Patient information included age, gender, medical comorbidities, symptoms, vital signs, laboratory and imaging findings at the time of presentation to hospital. Presence of keywords on imaging reports typical of COVID-19 infection (i.e., ground-glass, patchy infiltrates, opacification, airspace disease, consolidation, crazy paving sign) as reported in the literature, were used as a dichotomous variable [11] [15]. Admission date, discharge date, ICU admission, and mortality were also collected. This study was approved by the Research Ethics Board, University of British Columbia.

### Diagnostic testing

Specimens for testing included nasopharyngeal swab, sputum, tracheal aspirate, or bronchoalveolar lavage. Testing for viral RNA consisted of one of two commercial methods: LightMix® Real-Time PCR COVID-19 assay for the Envelope E-gene (TIB Molbiol, Germany) with amplification on the Roche LightCycler® 480, or Roche cobas® SARS-CoV-2 Qualitative Assay on the cobas® 6800 System for detection of ORF1/a and E-genes (Roche Molecular Diagnostics, Laval, QC). Antibody testing was performed on patients with serum collected ≥2 weeks and <4 months after PCR testing or symptom onset. Elecsys® Anti-SARS-CoV-2 assay (Roche) using recombinant protein representing the nucleocapsid (N) antigen for determination of total antibodies was performed on the Roche cobas® e601. The test result is given as a cut-off index (COI) with COI≥1.0 considered “reactive” and COI<1.0 as “non-reactive” as per package insert [16].

### Assessment of clinical likelihood of COVID-19

Patients were classified as having low, moderate, or high likelihood of COVID-19 based on chart review and clinical assessment [6] [7] [8]. The assessors were asked to review epidemiologic risk factors, clinical signs and symptoms, imaging findings, and laboratory results (other than SARS-CoV-2 PCR or serology results) to judge the probability of COVID-19 infection. Low probability cases had a clear alternative diagnosis explaining the clinical presentation and/or clinical features inconsistent with a COVID-19 infection; moderate probability cases had compatible clinical features and/or radiology findings but a presumptive alternative diagnosis; and high probability cases presented with compatible clinical features, radiological findings, no alternative diagnosis and/or an epidemiologic link to a known COVID-19 case. Patients deemed moderate to high risk for COVID-19 on initial assessment, underwent further review by an “expert panel” consisting of two internal medicine specialists caring for patients on dedicated COVID-19 hospital units. Any discordance in “expert” assessment was reviewed by an infectious disease specialist. Reviewers were not blinded to the PCR test result as it was reported in the electronic medical chart but were blinded to the serological result.

### Composite reference standard

The composite reference standard included the clinical assessment, any PCR result and serology. Positive reference standard was defined as testing positive on at least 2 out of 3 modalities (moderate or high clinical likelihood, PCR and serology). Composite reference testing was only performed on cases which had all 3 modalities, including serology, available. Cases with no serology available but PCR conversion from negative to positive for the same clinical episode within 4 weeks were deemed false negative on initial PCR and were included in the calculation of diagnostic accuracy. PCR reversion from positive to negative in patients with resolved clinical symptoms within the follow-up period was not considered false positive as such conversion was considered as part of the natural history of the disease.

### Statistical Analysis

The analysis population included all eligible patients who were admitted to hospital during the study period. Pearson’s chi-squared test was used to measure the association between binary or categorical variables and PCR result. T-tests were performed to compare the equality of means between continuous variables. For the sensitivity, specificity, positive and negative predictive values, exact binomial 95% confidence intervals were calculated. Stata version 14.1 was used for all analyses [17].

## Results

### Epidemiological data

There were 323 patients included in the study of which 33 (10.2%) tested positive and 290 (89.8%) tested negative on initial PCR. The mean age of the PCR positive and negative cases was 70.4 and 58.3, respectively (Table 1). Males and females comprised 69.3% and 30.3% of PCR negative cases and 66.7% and 33.3% of PCR positive cases, respectively. The average time from symptom onset to hospital admission in PCR positive and negative cases was 4.8 and 4.2 days, respectively. Of the PCR positive cases, 30.3% reported having a high risk exposure to a COVID-19 confirmed case and 12.1% reported a travel history, while only 1.7% of PCR negative cases had a high risk exposure and 3.1% had a travel history.

View this table:
[Table 1.](http://medrxiv.org/content/early/2021/02/22/2021.02.18.21252016/T1)

Table 1. Clinical Characteristics of the Study Patients According to PCR result

With regards to clinical features at presentation, PCR positive cases were more likely to have cough 66.7% vs 39.3%, shortness of breath 63.6% vs 35.9% and fever 72.7% vs 27.6%. Leukocytosis was present in 6.3% of PCR positive and 47.2% of PCR negative patients (p<0.001). Lymphopenia (≤1.1 ×109/L) was present in 66.7% of PCR positive and 49.7% of PCR negative cases (p=0.064).

Positive chest x-ray (CXR) findings were reported in 83.3% of the PCR positive cases versus 39.6% of PCR negative patients (p < 0.001). Positive Chest CT features were present in 100% of PCR positive cases compared to 45.3% of PCR negative cases (p=0.062). Of the PCR positive cases, 4 (12.9%) cases were mild, 14 (45.2%) were moderate, 5 (15.2%) were severe, and 8 (25.8%) were critical; 2 (6.1%) cases were unable to be classified.

There were 24.2% and 9.7% ICU admissions in the PCR positive and negative groups, respectively (p=0.012). Mortality occurred within 30 days in 21.2% of PCR positive and 6.2% of PCR negative cases (p=0.002) (Table 1).

All patients had an NP swab collected initially. Subsequent samples included repeat NP swabs (102), sputum (6), saliva (3), tracheal aspirate (2), bronchoalveolar lavage (3) and rectal swab (1). There were no cases that initially tested negative by NP that subsequently tested positive by an alternative sampling method. In two cases, an initial negative PCR was followed by a positive PCR test occurring during the same clinical episode (within the 30 days). These cases were coded as false negatives by initial PCR.

### Clinical assessment

On clinical assessment of PCR negative cases, 245, 37, and 8 were deemed low, moderate and high probability for COVID-19, respectively. Of PCR positive cases, 1, 3 and 29 were assigned low, moderate, and high probability, respectively. Inter-assessor reliability of cases with moderate or high clinical suspicion was 68.4% (k=0.68) [18]. In cases where composite standard was available, clinical assessment of high likelihood correctly identified 91.2% of true cases, whereas of cases with moderate and low clinical likelihood, 11.5% and 0.4% were true cases, respectively.

### PCR comparison with Serology

Serology was available for 90 (27.9%) of included patients, of which 17 tested “reactive” and 73 tested “non-reactive”. Serology was done at a mean of 69 days (range 14 to 138 days) from symptom onset. The agreement between PCR and Serology was 98.9%, with one PCR positive case testing non-reactive on serology (COI=0.73) (Table 3).

### PCR comparison with composite reference standard

There were 90 patients included in the composite reference standard: 19 (21.1%) positive, 71 (78.9%) negative. Compared to the composite reference standard initial PCR was found to have no false positives and 2 false negative. Sensitivity was estimated as 94.7% (95% CI 74.0 to 99.9%), specificity was 100% (95% CI 94.9 to 100%), positive predictive value was 100% (95% CI 81.5 to 100%) and a negative predictive value was 98.6% (95% CI 92.5 to 100%) (Table 2).

View this table:
[Table 2:](http://medrxiv.org/content/early/2021/02/22/2021.02.18.21252016/T2)

Table 2: Diagnostic accuracy of PCR compared to composite reference standard

View this table:
[Table 3:](http://medrxiv.org/content/early/2021/02/22/2021.02.18.21252016/T3)

Table 3: Comparison of PCR to Serology for SARS-CoV-2 diagnosis

## Discussion

In our study population, older age and male gender were associated with PCR positivity in keeping with previous reports of elderly and male patients having more severe disease requiring hospitalization [19]. Rates of epidemiological links to a known COVID-19 exposure were significantly higher in PCR positive cases at 31.4%. As previously reported, cough, fever, shortness of breath and myalgia were more frequently seen in PCR positive cases admitted to hospital [6]. Lymphopenia was present in 66.7% of PCR positive and 49.7% of PCR negative cases although this difference was not statistically significant (p=0.064). Observed trend of higher proportion of lymphopenia in PCR positive cases did not reach statistical significance perhaps as a result of high overall prevalence of lymphopenia in our study population due to other causes (HIV, solid organ transplantation, steroid therapy etc.) and small number of cases. Leukocytosis however, was significantly more frequent among PCR negative cases suggesting alternative diagnoses (i.e. bacterial causes of infection). Rates of severe outcomes including ICU admission of 24% and mortality of 21% were higher in the PCR positive group and similar to previous reports in hospitalized patients including early reports from Wuhan and more recent data from the UK [6] [20].

Positive CXR findings were more likely to be present in PCR positive cases compared to PCR negative cases (p<0.001); presence of positive CT chest finding did not reach statistical significance (p=0.095) between the two groups, perhaps due to limited numbers. Studies comparing the diagnostic accuracy of CT chest to PCR reported higher sensitivity but poor specificity (∼25%) of CT chest [21] [22] [10] [11] [23]. CT chest for primary screening or diagnosis of COVID-19 is not helpful in low prevalence setting due to significant rate of false positives [24]. Even though radiological findings may be helpful as an added diagnostic tool in settings of high COVID-19 incidence, they not only lack specificity but can also be falsely negative early after symptom onset [11] [15].

The estimated clinical sensitivity of NP swab PCR of 94.3% from this study echoes a study by Miller et al. who similarly ascertained PCR sensitivity of 95% in the first 5 days post-symptom onset in hospitalized patients [25]. Earlier studies have suggested clinical sensitivity of molecular assays to be in the range of 70% [26] [27] [28]. One of the earlier studies was a letter by Wang et al which included only 8 “nasal swabs” in their analysis [26]. While there is much interchangeable use of “nasal” and “nasopharyngeal (NP) swabs” in the literature, these are considerably different collection methods with “nasal swabs” being much less sensitive [29] [30] [31] [32]. Our group has further demonstrated the importance of proper specimen collection method with increased rate of false negative results in inadequately collected NP swabs [32]. Yang et al. (pending peer review) also reported low sensitivity of PCR, however they similarly included nasal swabs and utilized molecular assay with lower analytical sensitivity [27] [33] [34]. Although their study had limited data on patient characteristics and confirmation of COVID-19 diagnosis, it did suggest improved accuracy of lower respiratory tract specimens in diagnosis of more severe cases of COVID-19 [27]. Indeed pathogenesis of SARS-COV-2 demonstrates that the virus, once acquired, replicates in the upper respiratory tract and in a subset of patients advances and propagates in the lower respiratory airways and alveoli causing a more severe presentation [35]. However, there are practical challenges in obtaining lower respiratory tract specimens due to difficulty with sputum production or concerns of aerosol generation with bronchoscopy. In our study, only in a small proportion (3.4%) of patients had lower respiratory swabs collected.

There were 2 false negative cases on initial PCR converting to positive on subsequent testing. In the first case, the initial negative NP swab was taken 24 hrs after symptom onset and became positive 8 days later. PCR testing earlier than 48 hours of symptom onset can lead to false negative results as viral shedding can be below the level of detection [36]. The second false negative occurred in an elderly patient with severe viral pneumonia and high clinical likelihood of COVID-19. The patient’s first three NP swabs were negative until the fourth one tested positive 18 days after symptom onset. In severe infection viral load tends to be higher and peaks later; in this patient, it is possible that a lower respiratory specimen would have yielded better viral RNA recovery [37]. Clinical diagnostic accuracy of a SARS-CoV-2 PCR assay depends on the timing of presentation, clinical syndrome, anatomical site of testing, and quality of specimen collection, all of which are separate from the analytic performance of the assay itself [38] [32] [36].

As no accepted true gold standard for diagnosis of COVID-19 is available, we have developed a pragmatic composite reference model based on clinical assessment by medical experts, PCR and serology. A number of clinical prediction models have been proposed, however they have suffered from a high risk of bias and a lack of validation, therefore we opted against using them in our study [39]. Our model, although practical, is affected by the subjective nature of clinical likelihood assessment. Expert group inter-assessor reliability was 68.4% in assigning patients to moderate or high clinical likelihood groups. The moderate agreement could be reflective of non-specific nature of COVID-19 presentation as well as the risk attitudes of assessors [40]. In order to mitigate some of this subjectivity, we assigned patients deemed moderate or high likelihood category as positive on clinical assessment and cases in low likelihood category as negative. PCR and serology are incorporated as objective parts of the model; however, the evaluators were blinded to serology but not PCR results, which may have introduced bias in the clinical assessment.

Serological immunoassays have more recently become an adjunct to testing for COVID-19 [12] [41]. Similar to nucleic acid testing, serological diagnostic accuracy is determined by the target antigen and timing of collection. For example, antibodies (Abs) targeting the SARS-CoV-2 N protein are detectable earlier than antibodies against S protein, and Anti-N-protein IgG Abs tend to decrease earlier in the disease course than Anti-S-protein IgG Abs [42]. Our study criteria of serological inclusion is reflective of the fact that antibodies are more reliably detectable after two weeks and begin to decrease at four months after symptom onset [42] [12] [43] [41]. The correlation of PCR with serology was 98.9%, further increasing our confidence in the diagnostic performance on properly collected NP swabs. There were 2 false negative serology results compares to composite reference standard. While 99% of truly negative serological cases had COI value <0.1 (data not shown), one of the false negative cases had COI value of 0.73 closer to a cut-off for positivity. This case was a heart transplant recipient who was deemed a true case based on high clinical likelihood and a positive PCR. Perhaps due to his immunosuppression, this patient was unable to mount a strong immune response. Interestingly, another true COVID-19 case with prior renal transplant had a serology level at the threshold of positivity (COI=0.99) as well. The second false negative case was an elderly patient on rituximab for rheumatoid arthritis (COI=0.099). Recent studies have observed blunted or absent serological response in transplanted patients and persons taking immunosuppressive medication [44] [45]. Further work is needed in characterizing patients with low levels of IgG/IgM due various immunodeficiencies and interpretation of serological results in immunocompromised individuals should be done with caution.

The limitations of our study are the retrospective design and selection bias due to a hospital setting. Our data on clinical sensitivity applies to sicker, hospitalized patients who tend to have greater viral shedding, possibly leading to improved rates of PCR detection [46] [37]. Clinical assessment of COVID-19 likelihood was not blinded to PCR results as reviewers had access via an electronic medical record. Additionally, we have included the initial PCR test under evaluation as part of the composite reference standard. Even though we have expanded the reference standard to include any subsequent PCR results, this could have artificially enhanced the diagnostic performance of the index PCR and introduced a bias. The proposed reference standard has not been yet fully validated but reflects a practical approach utilizing currently available diagnostic modalities. The presence of a clear alternative diagnosis was used to assign lower likelihood of COVID-19 infection and although unlikely, there may have been cases of dual diagnoses. Furthermore, we were only able to obtain serology on a subset of patients as many were discharged from hospital prior to the time required to develop antibodies and had no subsequent bloodwork. Similar to PCR, false negative serology on patients could have overestimated PCR sensitivity, but in our data set, this is a rare occurrence which primarily applies to highly immunocompromised individuals.

## Conclusion

In summary, the risk of false-negative results with nucleic acid amplification tests is mostly related to pre-analytical factors such as timing of collection, the quality of sampling method and specimen type [36] [42]. Molecular testing on NP swabs has a high clinical sensitivity and excellent correlation with serology. As recommended by IDSA guidelines, cases with high clinical likelihood of COVID-19 and repeatedly negative NP swab PCR should undergo testing with serology to further enhance diagnostic yield [47], and a single PCR result cannot be interpreted in isolation without full clinical assessment of the case.

## Data Availability

Data was summarized in the Tables. Additional data needed can be made available upon request.

## Acknowledgements

The authors would like to thank Dr. Christopher Ryerson, Dr. Jane McKay and the Post-COVID-19 Recovery Clinic for their support of this work. We would like to extend our appreciation to Dr. Althea Hayden and Dr. Rohit Vijh for their guidance. We are also grateful to our medical microbiology and biochemistry technologists and scientists for their commitment to providing high quality of laboratory testing.

*   Received February 18, 2021.
*   Revision received February 18, 2021.
*   Accepted February 22, 2021.


*   © 2021, Posted by Cold Spring Harbor Laboratory

This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  [1]. Z. Wu and  J. M. McGoogan, “Characteristics of and Important Lessons from the Coronavirus Disease 2019 (COVID-19) Outbreak in China: Summary of a Report of 72 314 Cases from the Chinese Center for Disease Control and PRevention.,” JAMA, vol. 323(13), pp. 1239–1242, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.2648&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

2.  [2].WHO, “WHO Situation Report, #51”.
    
    

3.  [3].“[https://coronavirus.jhu.edu/map.html](https://coronavirus.jhu.edu/map.html),” [Online].
    
    

4.  [4].“[https://www.canada.ca/en/public-health/services/diseases/2019-novelcoronavirus-infection/guidance-documents/federal-provincial-territorial-public-health-response-plan-ongoing-management-covid-19.html](https://www.canada.ca/en/public-health/services/diseases/2019-novelcoronavirus-infection/guidance-documents/federal-provincial-territorial-public-health-response-plan-ongoing-management-covid-19.html),” [Online].
    
    

5.  [5]. R. T. Gandhi,  J. B. Lynch and  C. Del Rio, “Mild or Moderate Covid-19,” N Engl J Med, 2020.
    
    

6.  [6]. W. J. Guan,  Z. Y. Ni,  Y. Hu,  W. H. Liang,  C. Q. Ou,  J. X. He,  L. Liu,  H. Shan,  C. L. Lei,  D. Hui,  B. Du and et al., “China Medical Treatment Expert Group for Covid-19. Clinical Characteristics of Coronavirus Disease 2019 in China,” N Engl J Med, vol. 382, no. 18, pp. 1708–1720, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2002032&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

7.  [7]. D. Wang,  B. Hu,  C. Hu,  F. Zhu,  X. Liu,  J. Zhang and et al., “Clinical Characteristics of 138 Hospitalized Patients with 2019 Novel Coronavirus-Infected Penumonia in Wuhan, China,” JAMA, vol. 323, no. 11, pp. 1061–1069, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2020.1585&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

8.  [8]. C. Huang,  Y. Wang,  X. Li,  L. Ren and et al., “Clinical features of patients infected with 2019 nocel coronavirus in Wuhan, China,” Lancet, vol. 395, pp. 497–506, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(20)30183-5&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

9.  [9]. N. Zhu,  D. Zhang,  W. Wang,  X. Li,  B. Yang,  J. Song,  X. Zhao,  B. Huang,  W. Shi,  R. Lu,  P. Niu,  F. Zhan,  X. Ma,  D. Wang,  W. Xu,  G. Wu,  G. F. Gao and  W. Tan, “China Novel Coronaviurs Investigating and Research Team: A Novel Coronavirus from Patients with Pnumonia in China,” New England Journal of Medicine, vol. 382, no. 8, pp. 727–733, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa2001017&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

10. [10]. X. Xie,  Z. Zhong,  W. Zhao,  C. Zheng,  F. Wang and  J. Liu, “Chest CT for Typical 2019-nCoV Pneumonia: Relationship to Negative RT-PCR Testing.,” Radiology, vol. 12, 2020.
    
    

11. [11]. T. Ai,  Z. Yang,  H. Hou,  C. Zhan,  C. Chen,  W. Lv,  Q. Tao,  Z. Sun and  L. Xia, “Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases.,” Radiology, vol. 10, 2020.
    
    

12. [12]. Q. X. Long,  B. Z. Liu,  H. J. Deng and et al., “Antibody responses to SARS-CoV-2 in patients with COVID-19,” Nat Med, 2020.
    
    

13. [13].“[https://www.fda.gov/medical-devices/emergency-situations-medical-devices/emergency-use-authorizations](https://www.fda.gov/medical-devices/emergency-situations-medical-devices/emergency-use-authorizations),” 2020 April 2020. [Online].
    
    

14. [14]. F. Zhou,  T. Yu,  R. Du,  G. Fan,  Y. Lin and et al., “Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospectice cohort study,” Lancet, vol. 395, no. 10229, pp. 1054–1062, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0140-6736(20)30566-3&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

15. [15]. A. Bernheim,  X. Mei,  M. Huang and et al., “Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection.,” Radiology, vol. 295, no. 3, 2020.
    
    

16. [16].Roche-Canada, “Elecsys Anti-SARS-CoV-2,” [Online]. Available: [https://www.rochecanada.com/content/dam/rochexx/roche-ca/products/docs/package\_inserts/Can%20Elecsys%20Anti-SARS-CoV-2-09203079190-V1-PI-En-E2G.pdf](https://www.rochecanada.com/content/dam/rochexx/roche-ca/products/docs/package_inserts/Can%20Elecsys%20Anti-SARS-CoV-2-09203079190-V1-PI-En-E2G.pdf).
    
    

17. [17].Stata-Corp, “Stata statistical software: release 14.,” College Station, TX: StataCorp.
    
    

18. [18]. P. Brennan and  A. Silman, “Statistical methods for assessing observer variability in clinical measures,” BMJ, vol. 304, no. 6840, pp. 1491–1494, 1992.
    
    [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6MzoiUERGIjtzOjExOiJqb3VybmFsQ29kZSI7czozOiJibWoiO3M6NToicmVzaWQiO3M6MTM6IjMwNC82ODQwLzE0OTEiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMi8yMi8yMDIxLjAyLjE4LjIxMjUyMDE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

19. [19]. J. M. Jin,  P. Bai,  W. He,  F. Wu,  X. Liu,  D. M. Han,  S. Liu and  J. K. Yang, “Gender Differences in Patients with COVID-19: Focus on Severity and Mortality,” Front Public Health, vol. 8, no. 152, 2020.
    
    

20. [20]. N. Brendish,  S. Pool,  V. Naidu,  C. Mansbridge,  N. Norton and  H. Wheeler, “Clinical impact of molecular point-of-care testing for suspected COVID-19 in hospital (COV-19POC): a prospective, interventional, non-randomised, controlled study,” Lancet, vol. 8, no. 12, 2020.
    
    

21. [21]. C. Carver and  N. Jones, “Comparative accuracy of oropharyngeal and nasopharyngeal swabs for diagnosis of COVID-19,” CEBM, 2020.
    
    

22. [22]. L. Zou,  F. Ruan,  M. Huang and  L. Liang, “SARS-CoV-2 Viral Load in Upper Respiratory Specimens of Infected Patients,” N Engl J Med, vol. 382, no. 12, pp. 1177–1179, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMc2001737&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

23. [23]. Y. Fang,  H. Zhang,  J. Xie and et al., “Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR.,” Radiology, vol. 296, no. 2, pp. E115–E117, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200432&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

24. [24]. H. Kim,  H. Hong and  S. H. Yoon, “Diagnostic Performance of CT and Reverse Transcriptase Polymerase Chain Reaction for Coronavirus Disease 2019: A Meta-Analysis.,” Radiology, vol. 296, no. 3, pp. E145–E155, 2020.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

25. [25]. T. E. Miller,  W. F. Garcia,  A. Z. Bard,  T. Gogakos,  M. N. Anahtar,  M. G. Astudillo and et al., “Clinical sensitivity and interpretation of PCR and serological COVID-19 diagnostics for patients presenting to the hospital.,” FASEB J., 2020.
    
    

26. [26]. W. Wang,  Y. Xu,  R. Gao and et al., “Detection of SARS-CoV-2 in Different Types of Clinical Specimens,” JAMA, 2020.
    
    

27. [27]. Y. Yang,  M. Yang,  C. Shen and et al., “Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections.,” medRxiv, 2020.
    
    

28. [28]. S. Woloshin,  N. Patel and  A. S. Kesselheim, “False Negative Tests for SARS-CoV-2 Infection - Challenges and Implications.,” N eNgl J MEd, vol. 383, no. 6, 2020.
    
    

29. [29].WHO, “WHO guidelines for the collection of human specimens for laboratory diagnosis of avian influenza infection,” 2020.
    
    

30. [30]. H. Pere,  I. Podglajen,  M. Wack,  E. Flamarion,  T. Mirault,  G. Goudot,  C. Hauw-Berlemont,  L. Le,  E. Caudron,  S. Carrabin,  J. Rodary,  T. Ribeyre,  L. Belec and  D. Veyer, “Nasal swab sampling for SARs-CoV-2: A convenient alternative in time of nasopharyngeal swab shortage.,” J Clin Microbiol., 2020.
    
    

31. [31].“[https://www.cdc.gov/coronavirus/2019-ncov/lab/guidelines-clinical-specimens.html](https://www.cdc.gov/coronavirus/2019-ncov/lab/guidelines-clinical-specimens.html),” [Online].
    
    

32. [32]. N. N. Kinloch,  G. Ritchie,  C. J. Brumme and et al., “Suboptimal Biological Sampling as a Probable Cause of False-Negative COVID-19 Diagnostic Test Results.,” J Infect Dis, vol. 222, no. 6, pp. 899–902, 2020.
    
    

33. [33].“[https://www.who.int/diagnostics\_laboratory/eual/200722\_final\_pqpr\_eul\_0513\_200\_00\_sars\_cov2\_nucleic\_acid.pdf?ua=1](https://www.who.int/diagnostics\_laboratory/eual/200722\_final\_pqpr\_eul\_0513\_200\_00\_sars_cov2_nucleic_acid.pdf?ua=1),” [Online].
    
    

34. [34]. T. Pillonel,  V. Scherz,  K. Jaton,  G. Greub and  C. Bertelli, “Letter to the editor: SARS-CoV-2 detection by real-time RT-PCR.,” Euro Surveill., vol. 25, no. 21, 2020.
    
    

35. [35]. R. J. Mason, “Pathogenesis of COVID-19 from a cell biology perspective.,” Eur Respir J., vol. 55, no. 4, 2020.
    
    

36. [36]. L. M. Kucirkak,  S. A. Lauer,  O. Laeyvendecker,  D. Boon and  H. Lessler, “Variation in False-NEgative Rate of Reverse Transcriptase Polymerase Chain Reaction-Based SARS-CoV-2 Tests by Time Since Exposure.,” Ann Intern Med., vol. 173, no. 4, pp. 262–267, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/m20-1495&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

37. [37]. S. Zheng,  J. Fan,  F. Yu,  B. Feng and et al., “Viral load dynamics and disease severity in patients infected with SARS-CoV-2 in Zhejiang province, China, Janurary - March 2020: retrospective cohort study,” Br Med J, 2020.
    
    

38. [38]. X. He,  E. Lau,  P. Wu and et al., “Temporal dynamics in viral shedding and transmissibility of COVID-19.,” Nat Med, vol. 26, pp. 672–675, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/M20-3012&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

39. [39]. L. Wynants,  B. Van Calster,  G. S. Collins and et al., “Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal.,” BMJ, vol. 369, 2020.
    
    

40. [40]. A. R. Beyer,  B. Fasolo,  P. A. de Graeff and  H. Hillege, “Risk attitudes and personality traits predict perceptions of benefits and risks for medicinal products: a field study of European medical assessors,” Value Health, vol. 18, no. 1, pp. 91–99, 2015.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jval.2014.10.011&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25595239&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

41. [41]. S. K. Vashist, “In Vitro Diagnostic Assays for COVID-19: Recent Advances and Emerging Trends.,” Diagnostics (Basel), vol. 10, no. 4, p. 202, 2020.
    
    

42. [42]. G. Caruana,  A. Croxatto,  A. Coste and et al., “Diagnostic Strategies for SARS-CoV-2 infection and interpretation of microbiological results,” Clin Micriobiol Infect, vol. 26, no. 9, pp. 178–182, 2020.
    
    

43. [43]. J. Zhao,  Q. Yuan,  H. Wang and et al., “Antibody responses to SARS-CoV-2 in patients of novel coronavirus disease,” Clin Infect Dis., 2020.
    
    

44. [44]. P. D. Burbelo,  F. X. Riedo,  C. Morishima,  S. Rawlings,  D. Smith and  e. al., “Sensitivity in Detection of Antibodies to Nucleocapsiod and Spike Prtoetins of Severe Acute Respiratroy Syndrome Coronavirus 2 in Patients with Coronavirus Disease 2019,” J Infect Dis., vol. 222, no. 2, pp. 206–213, 2020.
    
    

45. [45]. L. R. Peterson,  S. Sami,  N. Vuong, MAT.,  P. Pathela and  e. al., “Lack of antibodies to SARS-CoV-2 in a large cohort of previously infected persons,” Clinical Infectious Diseases, 2020.
    
    

46. [46]. R. Wolfel,  V. M. Corman,  W. Guggemos and et al., “Virological assessment of hospitalized patients with COVID-2019,” Nature, vol. 581, pp. 465–469, 2020.
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41586-020-2196-x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2021%2F02%2F22%2F2021.02.18.21252016.atom) 

47. [47].I. S. P. Guidelines. [Online]. Available: [https://www.idsociety.org/practice-guideline/covid-19-guideline-serology/](https://www.idsociety.org/practice-guideline/covid-19-guideline-serology/).
    
    

48. [48]. V. M. Corman,  M. Kaiser,  R. Molenkamp,  A. Meijer,  D. Chu, Landt, O and et al., “DEtection of 2019 novel coronavirus (2019-nCoV) by realtime RT-PCR.,” Euro Serveill, vol. 25, 2020.
    
    

49. [49].C. f. D. C. a. Prevention, CDC 2019 Novel Coronavirus (2019-nCoV) Real-Time RT-PCR Diagnositc panel..
    
    

50. [50]. R. Jin,  M. A. Pettengill,  N. L. Hartnett,  H. E. Auerbach,  S. C. Peiper and  Z. Wang, “Commercial SARS-CoV-2 Molecular Assays: Superior Analytical Sensitivity of cobas SARS-CoV-2 Relative to NxTAG Cov Extended Panel and ID NOW COVID-19 Test,” Arch Pathol Lab Med, 2020.
    
    

51. [51]. J. K. Chen,  C. Y. Yip,  J. W. Chan and et al., “Clinical Performance of the Luminex NxTAG CoV Extended Panel for SARS-CoV-2 Detection in Nasopharyngeal Specimens from COVID-19 Patients in Hong Kong,” Journal of clinical microbiology, vol. 58, no. 8, 2020.
    
    

52. [52].“[https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas](https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/vitro-diagnostics-euas),” [Online].
    
    

53. [53]. S. A. Irving,  M. F. Vandermaus,  D. K. Shay and  E. A. Belongia, “Comparison of nasal and nasopharyngeal swabs for influenza detection in adults,” Clin Med Res., vol. 10, no. 4, pp. 215–218, 2012.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6ImNsaW5tZWRyZXMiO3M6NToicmVzaWQiO3M6ODoiMTAvNC8yMTUiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMi8yMi8yMDIxLjAyLjE4LjIxMjUyMDE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

54. [54]. R. T. Suhandynata,  M. A. Hoffman,  M. J. Kelner,  R. W. McLawhon,  S. L. Reed and  R. L. Fitzgerald, “Longitudinal Monitoring of SARS-CoV-2 IgM and IgG Seropositvity to Detect COVID-19,” J Appl Lab Med, 2020.
    
    

55. [55]. R. Li,  S. Pei,  B. Chen,  Y. Song,  T. Zhang,  W. Yang and  J. Shaman, “Substantial undocumented infection facilitates the rapid dissemination of novel coronavirus (SARS-CoV-2),” Science, vol. 368, no. 6490, pp. 489–493, 2020.
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6Mzoic2NpIjtzOjU6InJlc2lkIjtzOjEyOiIzNjgvNjQ5MC80ODkiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMS8wMi8yMi8yMDIxLjAyLjE4LjIxMjUyMDE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

56. [56]. M. Artesi,  S. Bontems,  P. Gobbels,  M. Franckh and et al., “A recurrent mutation at position 26,340 of SARS-CoV-2 is associated with failure of the E-gene qRT-PCR utilized in a commercial dual-target diagnostic assay.,” J Clin Microbiol, 2020.
    
    

57. [57].U. F. a. D. Administration, “EUA authorized serology test performance,” 2020. [Online]. Available: [https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/eua-authorized-serology-test-performance](https://www.fda.gov/medical-devices/coronavirus-disease-2019-covid-19-emergency-use-authorizations-medical-devices/eua-authorized-serology-test-performance).
    
    

58. [58]. I. Huang and  R. Pranata, “Lymphopenia in severe coronavirus disease - 2019 (COVID-19): systematic review and meta-analysis.,” J Intensive Care, vol. 8, no. 36, 2020.
    
    

59. [59].WHO, “WHO Emergency Use Assessment Coronavirus disease (COVID-19) IVDs,” [Online]. Available: [https://www.who.int/diagnostics\_laboratory/eual/200722\_final\_pqpr\_eul\_0513\_200\_00\_sars\_cov2\_nucleic\_acid.pdf?ua=1](https://www.who.int/diagnostics\_laboratory/eual/200722\_final\_pqpr\_eul\_0513\_200\_00\_sars_cov2_nucleic_acid.pdf?ua=1). [Accessed January 2021].