Abstract
Objectives DeepBleed is the first publicly available deep neural network model for the 3D segmentation of acute intracerebral hemorrhage (ICH) and intraventricular hemorrhage (IVH) on non-enhanced CT scans (NECT). The aim of this study was to evaluate the generalizability in an independent heterogenous ICH cohort and to improve the prediction accuracy by retraining the model.
Methods This retrospective study included patients from three European stroke centers diagnosed with acute spontaneous ICH and IVH on NECT between January 2017 and June 2020. Patients were divided into a training-, validation- and test cohort according to the initial study. Model performance was evaluated using metrics of dice score (DSC), sensitivity, and positive predictive values (PPV) in the original model (OM) and the retrained model (RM) for each ICH location. Students’ t-test was used to compare the DSC between the two models. A multivariate linear regression model was used to identify variables associated with the DSC. Pearson correlation coefficients (r) were calculated to evaluate the volumetric agreement with the manual reference (ground truth: GT). Intraclass correlation coefficients (ICC) were calculated to evaluate segmentation agreement with the GT compared to expert raters.
Results In total, 1040 patients were included. Segmentations of the OM had a median DSC, sensitivity, and PPV of 0.84, 0.79, and 0.93, compared to 0.83, 0.80, and 0.91 in the RM, adjusted p-values > 0.05. Performance metrics for infratentorial ICH improved from a median DSC of 0.71 for brainstem and 0.48 for cerebellar ICH in the OM to 0.77 and 0.79 in the RM. ICH volume and location were significantly associated with the DSC, p-values < 0.05. Volumetric measurements showed strong agreement with the GT (r > 0.90), p-value >0.05. Agreement of the automated segmentations with the GT were excellent (ICC ≥ 0.9, p-values <0.001), however worse if compared to the human expert raters (p-values <0.0001).
Conclusions DeepBleed demonstrated an overall good generalization in an independent validation cohort and location specific variances improved significantly after model retraining. Segmentations and volume measurements showed a strong agreement with the manual reference; however, the quality of segmentations was lower compared to human expert raters. This is the first publicly available external validation of the open-source DeepBleed network for spontaneous ICH introduced by Sharrock et al.
Competing Interest Statement
Tobias Penzkofer reports research agreements (no personal payments, outside of submitted work) with AGO, Aprea AB, ARCAGY-GINECO, Astellas Pharma Global Inc. (APGD), Astra Zeneca, Clovis Oncology, Inc., Dohme Corp, Holaira, Incyte Corporation, Karyopharm, Lion Biotechnologies, Inc., MedImmune, Merck Sharp, Millennium Pharmaceuticals, Inc., Morphotec Inc., NovoCure Ltd., PharmaMar S.A. and PharmaMar USA, Inc., Roche, Siemens Healthineers, and TESARO Inc., and fees for a book translation (Elsevier). All other authors have no relevant financial or non-financial interests to disclose.
Funding Statement
This study did not receive any funding
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee/IRB of Charité Universitä tsmedizin Berlin, Germany gave ethical approval for this work [protocol number EA1/035/20]
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present study are available upon reasonable request to the authors
