Pre-test probability for SARS-Cov-2-related Infection Score: the PARIS score
============================================================================

* On Behalf of the APHP/Universities/Inserm COVID-19 Research collaboration
* Tordjman Mickael
* Mekki Ahmed
* Mali D. Rahul
* Saab Ines
* Chassagnon Guillaume
* Guillo Enora
* Burns Robert
* Eshagh Deborah
* Beaune Sebastien
* Madelin Guillaume
* Bessis Simon
* Feydy Antoine
* Mihoubi Fadila
* Doumenc Benoit
* Carlier Robert-Yves
* Drape Jean Luc
* Revel Marie-Pierre

## Abstract

**Background** Diagnostic tests for SARS-CoV-2 infection (mostly RT-PCR and Computed Tomography) are not widely available in numerous countries, expensive and with imperfect performance

**Methods** This multicenter retrospective study aimed to determine a pre-test probability score for SARS-CoV-2 infection based on clinical and biological variables. Patients were recruited from emergency and infectious disease departments and were divided into a training and a validation cohort. Demographic characteristics, clinical symptoms, and results of blood tests (complete white blood cell count, serum electrolytes and CRP) were collected. The pre-test probability score was derived from univariate analyses between patients and controls, followed by multivariate binary logistic analysis to determine the independent variables associated with SARS-CoV-2 infection. Points were assigned to each variable to create the PARIS score. ROC curve analysis determined the area under the curve (AUC).

**Findings** One hundred subjects with clinical suspicion of SARS-CoV-2 infection were included in the training cohort, and 300 other consecutive individuals were included in the validation cohort. Low lymphocyte (<1·3 G/L), eosinophil (<0·06G/L), basophil (<0·04G/L) and neutrophil counts (<5G/L) were associated with a high probability of SARS-CoV-2 infection. No clinical variable was statistically significant. The score had a good performance in the validation cohort (AUC=0.889 (CI: [0.846–0.932]; STD=0.022) with a sensitivity and Positive Predictive Value of high-probability score of 80·3% and 92·3% respectively. Furthermore, a low-probability score excluded SARS-CoV-2 infection with a Negative Predictive Value of 99.5%.

**Interpretation** The PARIS score based on complete white blood cell count has a good performance to categorize the pre-test probability of SARS-CoV-2 infection. It could help clinicians avoid diagnostic tests in patients with a low-probability score and conversely keep on testing individuals with high-probability score but negative RT-PCR or CT. It could prove helpful in countries with a low-availability of PCR and/or CT during the current period of pandemic.

**Funding** None

**Evidence before this study** In numerous countries, large population testing is impossible due to the limited availability and costs of RT-PCR kits and CT-scan. Furthermore, false-negativity of PCR or CT as well as COVID-19 pneumonia mimickers on CT may lead to inaccurate diagnoses. Pre-test probability combining clinical and biological features has proven to be a particularly useful tool, already used in clinical practice for management of patients with a suspicion of pulmonary embolism.

**Added value of this study** This retrospective study including 400 patients with clinical suspicion of SARS-CoV-2 infection was composed of a training and a validation cohort. The pre-test probability score (PARIS score) determines 3 levels of probability of SARS-CoV2 infection based on white blood cell count (lymphocyte, eosinophil, basophil and neutrophil cell count).

**Implications of the available evidence** This pre-test probability may help to adapt SARS-CoV-2 infection diagnostic tests. The high negative predictive value (99·5%) of the low probability category may help avoid further tests, especially during a pandemic with overwhelmed resources. A high probability score combined with typical CT features can be considered sufficient for diagnosis confirmation.

## Introduction

Reverse Transcription Polymerase Chain Reaction (RT-PCR) is currently the test of reference to diagnose patients with the emerging SARS-CoV-2 infection1. However, in numerous countries including France, RT-PCR kits are not available enough to allow for large population testing. A definitive diagnosis is essential to control this disease so diagnostic methods need to be improved. Computed Tomography (CT)-scan has a reported high sensitivity2 but radiological signs can be delayed after disease onset, with up to 56% CT negativity in the first 3 days of symptomatic infection3. Moreover, performing large scale CT scanning during the current pandemic is made difficult by the need to apply rigorous disinfection protocols between patients4. The Fleischner Society recently published a Multinational Consensus Statement on the use of thoracic imaging based on different scenarios5. Imaging is indicated in case of moderate to severe disease manifestations, but not indicated in case of mild symptoms consistent with COVID-19 and no risk factor for disease progression. Even though CT-scan has a higher sensitivity than PCR (71–95% for PCR and 97–98% for CT), radiologists may still experience some difficulties for differentiating COVID-19 from non-COVID pneumonia6,7. False-negativity of PCR or CT as well as COVID-19 pneumonia mimickers on CT may lead to inaccurate diagnoses. Pre-test probability combining clinical and biological features has proven to be a very useful tool, already used in clinical practice for management of patients with a suspicion of pulmonary embolism8. Therefore, we aimed to derive a new prediction score based on clinical and biological variables, independently from subjective clinical evaluation by assessing a retrospective cohort of patients admitted to the emergency department (ED) of our institution for a clinical suspicion of SARS-Cov-2 infection. We validated this **P**re-test probability for S**AR**S-Cov-2 **I**nfection based on **S**coring (PARIS score) in a distinct multicenter cohort of patients admitted for the same reason in ED or infectious disease departments of 3 hospitals.

## Methods

This retrospective observational study has been approved by our local ethic committee (Institutional Review Board N° AAA-2020–08014), which waived the need for patient consent.

### Derivation cohort

We elaborated a diagnostic strategy for SARS-Cov-2 infection based on clinical and/or biological features to determine pre-test probability before RT-PCR or CT-scan. We decided to recruit a retrospective cohort of 100 patients with both RT-PCR and CT-scan results available with a 1:1 patient:control inclusion ratio from ED at Cochin Hospital (Paris, France) with a suspicion of SARS-Cov-2 infection: 50 consecutive infected patients and 50 consecutive controls. Exclusion criteria were: 1) absence of confirmed diagnosis (diagnosis still under investigation) (N=4); 2) lack of blood test including complete white blood cell count and serum electrolytes (N=6); 3) absence of reported clinical characteristics (N=2). Between March 15th and April 5th, 112 patients were screened for a clinically suspected SARS-Cov-2 infection. One hundred patients were finally included. Clinical examination was standardized at the ED. Demographic characteristics and comorbidities including high blood pressure, respiratory diseases (asthma, restrictive or chronic obstructive pulmonary disease (COPD)), immunodeficiency, kidney failure were recorded. Furthermore, clinical symptoms such as cough, fever, headache, diarrhea, anosmia, ageusia, oxygen desaturation were evaluated. Finally, biological tests including results of complete white blood cell count, serum electrolytes and C Reactive Protein (CRP) were analyzed. A final diagnosis of SARS-Cov-2 infection was retained if the patient had a confirmed diagnosis with both positive RT-PCR and CT-scan showing signs of COVID-19 pneumonia (usually bilateral and peripheral ground-glass and consolidated pulmonary opacities): since no test had a perfect diagnostic performance at the time of the study, we used in the derivation cohort a strict definition of patients and controls. Thus, controls were patients with both negative RT-PCR results and negative CT-scan (normal or with alternative diagnosis). We evaluated clinical and biological variables to find the ones correlated with SARS-Cov-2 infection. We only included basic biological tests, widely available and affordable, which are adapted for a pre-test probability score.

### Validation cohort

Our pre-test diagnostic score was validated on 300 consecutive subjects suspected of SARS-Cov-2 infection (different from those of the derivation cohort) with both RT-PCR and CT-scan results available, recruited from the ED and infectious disease departments of three different hospitals (Cochin Hospital, Paris; Ambroise Paré Hospital, Boulogne; Raymond Poincaré Hospital, Garches). Patients were confirmed to have SARS-Cov-2 infection when the RT-PCR result was positive. If the RT-PCR was negative, patients were considered as COVID+ in this cohort if CT-scan images evaluated by a senior radiologist specialized in thoracic imaging (>10 years of experience) were typical for COVID-19 pneumonia (N=19). The exclusion criterion was the unavailability of initial complete white blood cell count (N=9 of the 309 initially included patients).

### Statistical analysis

Univariate analyses were performed to select potential independent variables of SARS-Cov-2 infection to include in the multivariate analysis. The Mann Whitney U test was performed for comparison of continuous variables such as hemoglobin, and chi-square test was used to comparison of nominal variables. Exact p-values were computed for the Mann Whitney tests without any assumption of normality or asymptomatic approximation. A p-value of less than 0·05 was considered as statistically significant (two-tailed). The statistically significant variables were evaluated to find the optimal cut-offs to differentiate patients and controls based on receiver operating characteristic (ROC) curve analysis (upper left point of the graph) 9 (Supplementary Table S1). A binary logistic regression analysis was performed to measure the relationship between the categorial variable representing presence or absence of SARS-Cov-2 infection and the potential independent variables. Variables with non-statistical significance in the multivariate models were removed. Odds-Ratio (OR) were used to assign points for the pre-test probability score to the highest values (OR>10). The Hosmer-Lemeshow goodness of fit statistic10 and Nagelkerke’s R2 were used to assess the model validity11. We computed the pre-test probability score for each patient in the derivation cohort. We performed a second ROC curve analysis to determine the area under the curve (AUC) and to choose the cutoff values, in order to determine the low-probability group (<5% probability of SARS-Cov-2 infection), the intermediate probability-group and the high-probability group (>90% of probability of SARS-Cov-2 infection). The proportion of patients with SARS-Cov-2 infection in each group was evaluated to check the accuracy of the PARIS score.

We assessed the ability of the score to differentiate patients and controls in the validation cohort by a ROC curve analysis and computed the ROC curve. The prevalence of SARS-Cov-2 infection was calculated in the 3 probability groups (low, intermediate, and high probability score). SPSS software (version 24, Chicago, IL, USA) was used for statistical analysis.

## Results

### Derivation cohort

The clinical and biological characteristics of the patients and controls included in the derivation cohort are presented in Table 1.

View this table:
[Table 1:](http://medrxiv.org/content/early/2020/05/03/2020.04.28.20081687/T1)

Table 1: 
Characteristics of patients with SARS-CoV-2 infection (confirmed with both PCR and CT) and controls. QPulmonary disease: asthma, COPD or restrictive syndrome; Results are presented as means [standard deviation]

Fever/temperature at ER, oxygen saturation and myalgias were the clinical variables that significantly differed between patients and controls. Patients presented with significantly lower frequency of pulmonary diseases (asthma, COPD or restrictive lung disease) and immunodeficiency. Among blood test variables, we noted significantly lower lymphocyte, neutrophil, eosinophil, monocyte, and basophil cell counts in patients compared to controls. Additionally, platelet count was significantly lower in patients with SARS-Cov-2 infection. No difference between the 2 groups was found for serum electrolytes and CRP.

All these variables were included in the multivariate binary logistic regression analysis. Only low lymphocyte, basophil, eosinophil, and neutrophil cell counts were significantly associated with SARS-Cov-2 infection (Table 2). ROC curve analysis determined the best cut-offs for sensitivity and specificity (Supplementary Table 1). Points were assigned according to the odds-ratio: only lymphopenia (<1·3G/L) was associated with OR>10 (12·7) and 2 points were consequently assigned. The final score is presented in Table 3. The score was evaluated in the cohort of derivation (AUC=0·921; STD=0·027; CI= [0·867–0·974]).

View this table:
[Table 2:](http://medrxiv.org/content/early/2020/05/03/2020.04.28.20081687/T2)

Table 2: 
Binary logistic regression using descending Wald model; OR: Odd-Ratio; Nagelkerke R2=0.74; Hosmer-Lemeshow goodness of fit statistic: p=0.63

View this table:
[Table 3:](http://medrxiv.org/content/early/2020/05/03/2020.04.28.20081687/T3)

Table 3: 
Pre-test diagnostic probability of COVID-19 infection: PARIS score

### Validation cohort

Individuals included in the validation cohort had a mean age of 63 years (significantly older than those of the training cohort; p=0·007), with 172 being males and 128 females. There were 208 SARS-Cov-2 infected patients (69·3%) and 92 controls (30·7%). The pre-test probability score had a good performance in this cohort (AUC=0·889 (CI: [0·846–0·932]; STD=0·022). The sensitivity and specificity of the high-probability score (score 4 or 5) were 80.3% and 84.8% respectively (Table 4). Furthermore, Negative Predictive Value (NPV) of a low-probability score (score 0 or 1) was 99·5%. A low-probability score included 48·9% of controls (N=45/92). Many controls (N=33; 35·9%) presented with intermediate probability scores (score 2 to 4) but the NPV of combined low- and intermediate-probability scores was only 65·5%. Forty of the 208 SARS-Cov-2 infected patients (19·2%) from the cohort of validation had an intermediate score. No patient with SARS-Cov-2 infection had a score equal to 0 and there was only one infected patient with a score of 1. Only three of the 79 individuals with a score of 5 were not infected with SARS-Cov-2 (Positive Predictive Value of 96·2%). ROC curve for the validation cohort is presented in Figure 1.

![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2020/05/03/2020.04.28.20081687/F1.medium.gif)

[Figure 1:](http://medrxiv.org/content/early/2020/05/03/2020.04.28.20081687/F1)

Figure 1: 
Receiver Operating Characteristic (ROC) curve of the PARIS score for the validation cohort. Area Under the Curve= 0.889

View this table:
[Table 4:](http://medrxiv.org/content/early/2020/05/03/2020.04.28.20081687/T4)

Table 4: 
Performance in the validation cohort depending on the value of the PARIS score; PPV=Positive Predictive Value; NPV= Negative Predictive Value

### Misclassification and reclassification

Fourteen controls presented with a high-probability score (≥4). The explanation for their leukopenia was septic shock (N=3), immunodepression induced by lupus (N=2), cystic fibrosis with lung transplantation (N=2), pancytopenia due to hematological disorder (N=2), chemotherapy for lymphoma (N=1). Four controls didn’t have any final diagnosis explaining their clinical symptoms and leukopenia and could therefore be false negative for SARS-Cov-2 infection. Only one patient was considered as a false-negative of the pre-test probability score with a low-probability score of 1. This patient had negative RT-PCR and minimal involvement on CT-scan (small ground glass opacities).

However, the score was also helpful to reclassify patients: Six out of 10 infected patients, based on RT-PCR, with negative or indeterminate CT-scan results had a high-probability score (while the 4 other patients had an intermediate PARIS score). Fifteen out of 21 patients with negative PCR but CT features typical for COVID-19 pneumonia had a high-probability score (Five other patients had an intermediate score and one patient had a low score).

## Discussion

In this study, we established a pre-test probability score (PARIS score) based on simple, affordable, and widely available blood tests (complete white blood cell count). No clinical variable was independently associated with SARS-Cov-2 infection but four biological variables(lymphocytes<1·3G/L,eosinophils<0·06G/L,basophiles<0·04G/L, neutrophils<5G/L) were associated to the condition. This finding could be particularly useful for countries with a low availability of diagnostic tests.

A low lymphocyte count has been described in patients with SARS-Cov-2 infection, possibly due to a decreased level of T-lymphocytes12. Furthermore, the total number of NK and CD8+ T cells has been reported to be markedly decreased in patients with SARS-Cov-2 infection13. Decreased absolute number of T lymphocytes is more pronounced in severe cases while decreased CD8+ T cells count could be an independent variable associated with the severity of the disease14–16. Eosinophils have also been reported to be decreased in previous studies17,18. Increasing eosinophil count may even be a predictor of improvement in patients with SARS-Cov-2 infection19. A low proportion of basophils has also been reported in severe cases12 but no previous study has reported low basophil cell count as a marker of SARS-Cov-2 infection. Furthermore, previous reports demonstrated low levels of neutrophils in SARS-Cov-2 patients20 but increased neutrophil count in severe cases of COVID-19 pneumonia21. Two previous histological studies of autopsies performed in COVID-19 patients showed inflammatory cells including lymphocytes and macrophages in the alveoli, with minimal eosinophilic and neutrophilic infiltration22,23. Decreased lymphocyte blood cell count could therefore be due to lymphocytic lung infiltration.

The Fleischner Society recently released a report highlighting that no score was available to assess pre-test probability in patients with mild symptoms consistent with SARS-Cov-2 infection5. This probability is only based on background prevalence of the disease and individual’s exposure risk. Committee members disapproved imaging use in case of mild symptoms and a negative COVID-19 test. The PARIS score may enhance the proposed Fleischner diagnostic algorithm by improving the pre-test performance. Thus, in case of high pre-test probability with no comorbidities explaining leukopenia, a second RT-PCR test or an additional CT-scan may be useful to establish the final diagnosis. Furthermore, a low PARIS score may obviate the need for RT-PCR or CT-scan, especially in a period of diagnostic test shortage.

A previous review indicated that prediction models for diagnosis and prognosis of COVID-19 infection are poorly reported, at high risk of bias, and with reported performance probably too optimistic24. We report here a multicenter study assessing diagnostic pre-test probability with a good accuracy, using a study design similar to that of the revised Geneve Score for pulmonary embolism8.

Clinical symptoms were not included in our model, since most of patients and controls presented with the same symptomatology, with no specificity of symptoms. Even oxygen saturation was not a discriminative feature. It could be explained by the fact that most patients with SARS-Cov2 infection were not severe, with subnormal oxygen saturation. Furthermore, even if patients with lung diseases were more frequent in the control group, their average oxygen saturation was 96%. Thus, low oxygen saturation may be an additional argument of SARS-Cov-2 infection in patients with no previous pulmonary disease. However, the PARIS score is mostly useful in patients with relatively mild clinical presentation. It could also be useful for initial triage of patients presenting to the ED because it is fast, affordable and allows patients with a low probability score to stay apart from those most likely to be infected, avoiding the risk of contamination.

Our study presents inherent limitations. First, its retrospective design may introduce a selection bias. Only 100 patients were included in the derivation cohort in order to recruit a similar number of infected patients and controls (50 in each group). There was no statistically significant difference between demographic characteristics of patients and controls but the 2 groups were not matched and controls more frequently presented pulmonary diseases and immunodeficiencies. A small number of patients was included due to the urgent need of diagnostic tools, but this is at risk of low reproducibility in a larger population. Indeed, the performance of the pre-test score was slightly lower in the validation cohort. We used only regular blood tests with complete white blood cell count, serum electrolytes and CRP, and did not evaluate LDH or D-dimers, since they seem to be mainly correlated with severity and have been described as risk factors for acute respiratory distress syndrome25. Another limit of this study is the only inclusion of patients from hospital wards, which probably induces a recruitment bias, since they may not be representative of patients with milder symptoms who consult their general practitioner, even though most symptomatic patients are directly referred to ED in our country. The evaluation of the performance of our pre-test probability score in a larger population would be of interest. Even though a part of patients in our cohort were admitted to Intensive Care Units (ICUs) shortly after presentation, more severe patients directly admitted in ICUs, may be underrepresented. However, the score might not be as useful for severe patients admitted to ICU, since they require admission and urgent care, regardless of RT-PCR positivity. This score was also built during a pandemic with high prevalence of the disease and might not be as efficient in a situation where SARS-Cov2 would be less circulating. Numerous diseases or conditions may mimic SARS-Cov-2 biological changes with leukopenia and lymphopenia. Clinicians should be aware of the limits of this score in case of hematological disease history. Lymphocyte count may be high in infected patients with leukemia while myelodysplasia may falsely induce a high-probability score. This was observed in a patient with typhoid fever, inducing a high probability score in the cohort of derivation.

In conclusion, the PARIS score shows good performance with high positive predictive value for intermediate or high score and high negative predictive value to exclude SARS-Cov-2 infection in low score individuals. It may help avoid useless repeated tests in case of low probability, especially during a pandemic with no possibility for large scale testing. Finally, a high score with typical CT features can be considered as sufficient to confirm the diagnosis.

## Data Availability

All data from this manuscript are available

View this table:
[Supplementary Table 1:](http://medrxiv.org/content/early/2020/05/03/2020.04.28.20081687/T5)

Supplementary Table 1: 
Receiver Operator characteristic (ROC) curve analyses to determine the adequate cut-off for the independent variables; AUC= Area Under the Curve

### Authors’ contribution

Study conception and design: Tordjman, Revel

Acquisition of data: Mekki, Saab, Chassagnon, Guillo, Doumenc, Mihoubi, Feydy, Bessis, Beaune

Analysis and interpretation of data: Mali, Madelin, Tordjman, Mekki, Eshagh

Drafting of manuscript: Tordjman, Mekki, Saab, Mihoubi, Burns, Mali, Eshagh, Revel

Critical revision: Chassagnon, Mihoubi, Madelin, Eshagh, Beaune, Bessis, Doumenc, Feydy, Carlier, Drapé, Revel

All authors discussed the results and commented on the manuscript. All authors gave final approval of the version to be submitted.

## Acknowledgments

We acknowledge APHP, universities, Inserm for COVID-19 research collaboration. We acknowledge Dr Martin, Dr Alperin, Pr Lee, Pr Allain and all doctors from the Emergency Department, ICU and hospitalization departments at Cochin Hospital.

## Footnotes

*   Funding: None

*   Conflict of interest: None

*   Received April 28, 2020.
*   Revision received April 28, 2020.
*   Accepted May 3, 2020.


*   © 2020, Posted by Cold Spring Harbor Laboratory

The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission.

## References

1.  1.Corman VM, Landt O, Kaiser M, et al. Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR. Euro Surveill. 2020;25(3). doi:10.2807/1560-7917.ES.2020.25.3.2000045
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2807/1560-7917.ES.2020.25.3.2000045&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31992387&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

2.  2.Ai T, Yang Z, Hou H, et al. Correlation of Chest CT and RT-PCR Testing in Coronavirus Disease 2019 (COVID-19) in China: A Report of 1014 Cases. Radiology. February 2020:200642. doi:10.1148/radiol.2020200642
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200642&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32101510&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

3.  3.Bernheim A, Mei X, Huang M, et al. Chest CT Findings in Coronavirus Disease-19 (COVID-19): Relationship to Duration of Infection. Radiology. February 2020:200463. doi:10.1148/radiol.2020200463
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200463&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32077789&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

4.  4.Nakajima K, Kato H, Yamashiro T, et al. COVID-19 pneumonia: infection control protocol inside computed tomography suites. Jpn J Radiol. March 2020. doi:10.1007/s11604-020-00948-y
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11604-020-00948-y&link_type=DOI) 

5.  5.Rubin GD, Ryerson CJ, Haramati LB, et al. The Role of Chest Imaging in Patient Management during the COVID-19 Pandemic: A Multinational Consensus Statement from the Fleischner Society. Radiology. April 2020:201365. doi:10.1148/radiol.2020201365
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020201365&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32255413&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

6.  6.Bai HX, Hsieh B, Xiong Z, et al. Performance of radiologists in differentiating COVID-19 from viral pneumonia on chest CT. Radiology March 2020:200823. doi:10.1148/radiol.2020200823
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200823&link_type=DOI) 

7.  7.Fang Y, Zhang H, Xie J, et al. Sensitivity of Chest CT for COVID-19: Comparison to RT-PCR. Radiology. February 2020:200432. doi:10.1148/radiol.2020200432
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1148/radiol.2020200432&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32073353&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

8.  8.Le Gal G, Righini M, Roy P-M, et al. Prediction of pulmonary embolism in the emergency department: the revised Geneva score. Ann Intern Med. 2006;144(3):165–171. doi:10.7326/0003-4819-144-3-200602070-00004
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/0003-4819-144-3-200602070-00004&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16461960&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000235074800002&link_type=ISI) 

9.  9.Fawcett T. An introduction to ROC analysis. Pattern Recognition Letters. 2006;27(8):861–874. doi:10.1016/j.patrec.2005.10.010
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.patrec.2005.10.010&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000237462800002&link_type=ISI) 

10. 10.Hosmer DW, Lemesbow S. Goodness of fit tests for the multiple logistic regression model. Communications in Statistics - Theory and Methods. 1980;9(10):1043–1069. doi:10.1080/03610928008827941
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1080/03610928008827941&link_type=DOI) 

11. 11.Nagelkerke NJD. A note on a general definition of the coefficient of determination. Biometrika. 1991;78(3):691–692. doi:10.1093/biomet/78.3.691
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/biomet/78.3.691&link_type=DOI) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1991GJ30600024&link_type=ISI) 

12. 12.Qin C, Zhou L, Hu Z, et al. Dysregulation of immune response in patients with COVID-19 in Wuhan, China. Clin Infect Dis. March 2020. doi:10.1093/cid/ciaa248
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciaa248&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32161940&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

13. 13.Zheng M, Gao Y, Wang G, et al. Functional exhaustion of antiviral lymphocytes in COVID-19 patients. Cell Mol Immunol. March 2020. doi:10.1038/s41423-020-0402-2
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41423-020-0402-2&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32203188&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

14. 14.Chen G, Wu D, Guo W, et al. Clinical and immunologic features in severe and moderate Coronavirus Disease 2019. J Clin Invest. March 2020. doi:10.1172/JCI137244
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1172/JCI137244&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32217835&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

15. 15.Wang F, Nie J, Wang H, et al. Characteristics of peripheral lymphocyte subset alteration in COVID-19 pneumonia. J Infect Dis. March 2020. doi:10.1093/infdis/jiaa150
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/infdis/jiaa150&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32227123&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

16. 16.Ji D, Zhang D, Xu J, et al. Prediction for Progression Risk in Patients with COVID-19 Pneumonia: the CALL Score. Clin Infect Dis. doi:10.1093/cid/ciaa414
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciaa414&link_type=DOI) 

17. 17.Li YX, Wu W, Yang T, et al. [Characteristics of peripheral blood leukocyte differential counts in patients with COVID-19]. Zhonghua Nei Ke Za Zhi. 2020;59():E003. doi:3760.10/cmaj.cn112138-20200221-00114
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=3760.10/cmaj.cn112138-20200221-00114&link_type=DOI) 

18. 18.Zhang J-J, Dong X, Cao Y-Y, et al. Clinical characteristics of 140 patients infected with SARS-CoV-2 in Wuhan, China. Allergy. February 2020. doi:10.1111/all.14238
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/all.14238&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

19. 19.Liu F, Xu A, Zhang Y, et al. Patients of COVID-19 may benefit from sustained lopinavir-combined regimen and the increase of eosinophil may predict the outcome of COVID-19 progression. Int J Infect Dis. March 2020. doi: 10.1016/j.ijid.2020.03.013
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ijid.2020.03.013&link_type=DOI) 

20. 20.Liu Y, Yang Y, Zhang C, et al. Clinical and biochemical indexes from 2019-nCoV infected patients linked to viral loads and lung injury. Sci China Life Sci. 2020;63(3):364–374. doi:10.1007/s11427-020-1643-8
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11427-020-1643-8&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32048163&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

21. 21.Mo P, Xing Y, Xiao Y, et al. Clinical characteristics of refractory COVID-19 pneumonia in Wuhan, China. Clin Infect Dis. March 2020. doi:10.1093/cid/ciaa270
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cid/ciaa270&link_type=DOI) 

22. 22.Barton LM, Duval EJ, Stroberg E, Ghosh S, Mukhopadhyay S. COVID-19 Autopsies, Oklahoma, USA. Am J Clin Pathol. April 2020. doi:10.1093/ajcp/aqaa062
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/ajcp/aqaa062&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32275742&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

23. 23.Yao XH, Li TY, He ZC, et al. [A pathological report of three COVID-19 cases by minimally invasive autopsies]. Zhonghua Bing Li Xue Za Zhi. 2020;49():E009. doi: 10.3760/cma.j.cn112151-20200312-00193
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.3760/cma.j.cn112151-20200312-00193&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom) 

24. 24.Wynants L, Van Calster B, Bonten MMJ, et al. Prediction models for diagnosis and prognosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369:m1328. doi:10.1136/bmj.m1328
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE3OiIzNjkvYXByMDdfMi9tMTMyOCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDIwLzA1LzAzLzIwMjAuMDQuMjguMjAwODE2ODcuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

25. 25.Wu C, Chen X, Cai Y, et al. Risk Factors Associated With Acute Respiratory Distress Syndrome and Death in Patients With Coronavirus Disease 2019 Pneumonia in Wuhan, China. JAMA Intern Med. March 2020. doi:10.1001/jamainternmed.2020.0994
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamainternmed.2020.0994&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32167524&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2020%2F05%2F03%2F2020.04.28.20081687.atom)