Abstract
Background In the midst of the coronavirus disease 2019 (COVID-19) outbreak, chest X-ray (CXR) imaging is playing an important role in the diagnosis and monitoring of patients with COVID-19. Machine learning solutions have been shown to be useful for X-ray analysis and classification in a range of medical contexts.
Purpose The purpose of this study is to create and evaluate a machine learning model for diagnosis of COVID-19, and to provide a tool for searching for similar patients according to their X-ray scans.
Materials and Methods In this retrospective study, a classifier was built using a pre-trained deep learning model (ReNet50) and enhanced by data augmentation and lung segmentation to detect COVID-19 in frontal CXR images collected between January 2018 and July 2020 in four hospitals in Israel. A nearest-neighbors algorithm was implemented based on the network results that identifies the images most similar to a given image. The model was evaluated using accuracy, sensitivity, area under the curve (AUC) of receiver operating characteristic (ROC) curve and of the precision-recall (P-R) curve.
Results The dataset sourced for this study includes 2362 CXRs, balanced for positive and negative COVID-19, from 1384 patients (63 +/- 18 years, 552 men). Our model achieved 89.7% (314/350) accuracy and 87.1% (156/179) sensitivity in classification of COVID-19 on a test dataset comprising 15% (350 of 2326) of the original data, with AUC of ROC 0.95 and AUC of the P-R curve 0.94. For each image we retrieve images with the most similar DNN-based image embeddings; these can be used to compare with previous cases.
Conclusion Deep Neural Networks can be used to reliably classify CXR images as COVID-19 positive or negative. Moreover, the image embeddings learned by the network can be used to retrieve images with similar lung findings.
Summary Deep Neural Networks and can be used to reliably predict chest X-ray images as positive for coronavirus disease 2019 (COVID-19) or as negative for COVID-19.
Key Results
A machine learning model was able to detect chest X-ray (CXR) images of patients tested positive for coronavirus disease 2019 with accuracy of 89.7%, sensitivity of 87.1% and area under receiver operating characteristic curve of 0.95.
A tool was created for finding existing CXR images with imaging characteristics most similar to a given CXR, according to the model’s image embeddings.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
The work was funded by CoronaVirus Fund, Weizmann Insititue of Science.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
This retrospective study was approved by the Institutional Review Board (IRB) and the Helsinki committee of the participating medical centers: 1) Department of Radiology, HaEmek Medical Center, Afula, Israel 2) Department of Otolaryngology, Head and Neck Surgery, Galilee Medical Center, Nahariya, Israel; 3) Cardiothoracic Imaging Unit, Shaare Zedek Medical Center, Jerusalem, Israel 4) Radiology department, Rabin Medical Center, Jabotinsky Rd 39, Petah Tikva; The study was approved in compliance with the public health regulations and provisions of the current harmonized international guidelines for good clinical practice (ICH-GCP) and in accordance with Helsinki principles. Informed consent was waived by the IRB of the above centers for the purpose of this study. Data extracted from medical records retrieved included only non-identifying information such as age, sex, vital signs, blood counts, chemistry, SARS-CoV-2 swab testing results, chemistry, and X-ray imaging files obtained as part of the diagnostic pipeline upon admission and on routine medical follow-up.
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The code will be made available upon publication of the paper
Abbreviations
- CXR
- chest x-ray
- COVID-19
- Coronavirus Disease 2019
- RT-PCR
- reverse transcription polymerase chain reaction
- ROC
- receiver operating characteristic
- P-R curve
- precision-recall curve
- AUC
- area under the curve
- GT
- ground truth
- FPR
- false positive rate
- TPR
- true positive rate
