Abstract
Model initialization techniques are vital for improving the performance and reliability of deep learning models in medical computer vision applications. While much literature exists on non-medical images, the impacts on medical images, particularly chest X-rays (CXRs) are less understood. Addressing this gap, our study explores three deep model initialization techniques: Cold-start, Warm-start, and Shrink and Perturb start, focusing on adult and pediatric populations. We specifically focus on scenarios with periodically arriving data for training, thereby embracing the real-world scenarios of ongoing data influx and the need for model updates. We evaluate these models for generalizability against external adult and pediatric CXR datasets. We also propose novel ensemble methods: F-score-weighted Sequential Least-Squares Quadratic Programming (F-SLSQP) and Attention-Guided Ensembles with Learnable Fuzzy Softmax to aggregate weight parameters from multiple models to capitalize on their collective knowledge and complementary representations. We perform statistical significance tests with 95% confidence intervals and p-values to analyze model performance. Our evaluations indicate models initialized with ImageNet-pretrained weights demonstrate superior generalizability over randomly-initialized counterparts, contradicting some findings for non-medical images. Notably, ImageNet-pretrained models exhibit consistent performance during internal and external testing across different training scenarios. Weight-level ensembles of these models show significantly higher recall (p<0.05) during testing compared to individual models. Thus, our study accentuates the benefits of ImageNet-pretrained weight initialization, especially when used with weight-level ensembles, for creating robust and generalizable deep learning solutions.
Author Summary In this research, we actively explore various techniques for optimal initialization of deep learning models for analyzing medical images such that the resulting models are generalizable and also demonstrate high performance. Generalizability is an area of significant importance. It is often ignored in favor of the model achieving high performance at the cost of maintaining on previously unseen, i.e. external data, that may also be out-of-distribution. This may result in the classifier performing inadequately thereby reducing its value. We demonstrate that unlike general-purpose images, such as those found in ImageNet collection, medical images, such as chest X-rays (CXRs), are different in their visual characteristics. We show that counter to previously reported results using non-medical images, ImageNet pre-trained models trained on medical images, in fact, converge sooner and generalize better than randomly initialized models. We compare three distinct model initialization methods using internal adult CXR data to train the models which are subsequently tested on external CXR images for both adult and pediatric populations. Additionally, we consolidate several of these models into an ‘ensemble’ to demonstrate that they achieve a more accurate identification of relevant cases during both internal and external testing. Therefore, our work underscores the promising potential of employing ImageNet-pretrained models for medical imagess and merging them into ensembles, aiming to enhance the reliability of AI in medical image analysis.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This work was supported by the Intramural Research Program of the National Library of Medicine (NLM), National Institutes of Health (NIH).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
All data used in this manuscript are deidentified and publicly available and exempted from IRB review.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
↵# Note: Dr. Ghada Zamzmi is currently a Staff Fellow at the Center for Devices and Radiological Health at the Food and Drug Administration.
Data Availability
All data used in this work are sourced from publicly available repositories. Details are included in the manuscript.
