Abstract
Objectives We tested artificial intelligence (AI) to support the diagnosis of COVID-19 using chest X-ray (CXR). Diagnostic performance was computed for a system trained on CXRs of Italian subjects from two hospitals in Lombardy, Italy.
Methods We used for training and internal testing an ensemble of ten convolutional neural networks (CNNs) with mainly bedside CXRs of 250 COVID-19 and 250 non-COVID-19 subjects from two hospitals. We then tested such system on bedside CXRs of an independent group of 110 patients (74 COVID-19, 36 non-COVID-19) from one of the two hospitals. A retrospective reading was performed by two radiologists in the absence of any clinical information, with the aim to differentiate COVID-19 from non-COVID-19 patients. Real-time polymerase chain reaction served as reference standard.
Results At 10-fold cross-validation, our AI model classified COVID-19 and non COVID-19 patients with 0.78 sensitivity (95% confidence interval [CI] 0.74–0.81), 0.82 specificity (95% CI 0.78–0.85) and 0.89 area under the curve (AUC) (95% CI 0.86–0.91). For the independent dataset, AI showed 0.80 sensitivity (95% CI 0.72–0.86) (59/74), 0.81 specificity (29/36) (95% CI 0.73–0.87), and 0.81 AUC (95% CI 0.73– 0.87). Radiologists’ reading obtained 0.63 sensitivity (95% CI 0.52–0.74) and 0.78 specificity (95% CI 0.61–0.90) in one centre and 0.64 sensitivity (95% CI 0.52–0.74) and 0.86 specificity (95% CI 0.71–0.95) in the other.
Conclusions This preliminary experience based on ten CNNs trained on a limited training dataset shows an interesting potential of AI for COVID-19 diagnosis. Such tool is in training with new CXRs to further increase its performance.
Key points
Artificial intelligence based on convolutional neural networks was preliminary applied to chest-X-rays of patients suspected to be infected by COVID-19.
Convolutional neural networks trained on a limited dataset of 250 COVID-19 and 250 non-COVID-19 were tested on an independent dataset of 110 patients suspected for COVID-19 infection and provided a balanced performance with 0.80 sensitivity and 0.81 specificity.
Training on larger multi-institutional datasets may allow this tool to increase its performance.
Competing Interest Statement
F. Sardanelli has received research grants from and is member of speakers’ bureau and of advisory group for General Electric, Bayer, and Bracco. Isabella Castiglioni, Matteo Interlenghi and Annalisa Polidori own DeepTrace Technologies S.R.L shares. Christian Salvatore is CEO of DeepTrace Technologies S.R.L. DeepTrace Technologies S.R.L is a spin-off of Scuola Universitaria Superiore IUSS, Pavia, Italy. The other authors of this manuscript declare no relationships with any companies, whose products or services may be related to the subject matter of the article.
Funding Statement
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. This study was partially supported by funding from the Italian Ministry of Health to IRCCS Ospedale San Raffaele.
Author Declarations
All relevant ethical guidelines have been followed; any necessary IRB and/or ethics committee approvals have been obtained and details of the IRB/oversight body are included in the manuscript.
Yes
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
E-mail: isabella.castiglioni{at}unimib.it, E-mail: davide.ippolito{at}unimib.it; d.gandola1{at}campus.unimib.it, E-mail: matteo.interlenghi{at}ibfm.cnr.it, E-mail: caterina.monti{at}unimi.it; francesco.sardanelli{at}unimi.it, E-mail: salvatore{at}deeptracetech.com; polidori{at}deeptracetech.com, E-mail: schiaffino.simone{at}gmail.com, E-mail: cristina.messa{at}unimib.it
Data Availability
Data used in this work are not publicly available.
Abbreviations
- RT-PCR
- Polymerase chain reaction
- CT
- Computed tomography
- GGOs
- Ground-glass opacities
- CXR
- Chest X-ray
- AI
- Artificial intelligence
- AUC
- Area under the curve
- CNN
- Convolutional neural network
