ABSTRACT
Objective To assess a deep learning-based artificial intelligence model for the detection of pulmonary nodules on chest radiographs and to compare its performance with board-certified human readers.
Methods For this retrospective study, 308 chest radiographs were obtained between January 2019 to December 2021 from a tertiary care hospital. All radiographs were analyzed using a deep learning AI model called DxNodule AI Screen. Two expert board-certified radiologists established the ground truth, and 11 test readers independently reviewed all radiographs in two sessions (unaided and AI-aided mode) with a washout period of one month.
Results The standalone model had an AUROC of 0.905 [0.87, 0.94] in detecting pulmonary nodules. The mean AUROC across the 11 readers improved from 0.798 [0.74, 0.86] for unaided interpretation to 0.846 [0.82, 0.880] for AI-aided interpretation. With DxNodule AI Screen, readers were able to identify nodules at the correct locations, which they otherwise missed. The mean specificity, accuracy, PPV, and NPV of the readers improved significantly from 0.87 [0.78, 0.96], 0.78 [0.72, 0.84], 0.77 [0.65, 0.88], and 0.86 [0.81, 0.90] in the unaided session to 0.89 [0.82, 0.96], 0.83 [0.80, 0.85], 0.82 [0.73, 0.9], and 0.89 [0.86, 0.92], respectively in the aided session.
Conclusion DxNodule AI Screen outperformed human readers in nodule detection performance on chest radiographs, and enhanced human readers’ performances when used as an aid.
Competing Interest Statement
Amit Kharat is a Professor at Dr. DY Patil Medical College, Pune, and is also a co-founder of DeepTek Medical Imaging Pvt Ltd. Vinay Duddalwar is a Professor of Radiology at Keck School of Medicine, USC, USA, and is also on the advisory board of DeepTek Medical Imaging Pvt Ltd. He is a consultant to Radmetrix Inc, Cohere Inc, and Westat Inc.
Funding Statement
This study did not receive any funding
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee of Dr. D.Y. Patil hospital and research center gave ethical approval for this work
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
The Chan Zuckerberg Initiative, Cold Spring Harbor Laboratory, the Sergey Brin Family Foundation, California Institute of Technology, Centre National de la Recherche Scientifique, Fred Hutchinson Cancer Center, Imperial College London, Massachusetts Institute of Technology, Stanford University, University of Washington, and Vrije Universiteit Amsterdam.