PT  - JOURNAL ARTICLE
AU  - Boyd, Aidan
AU  - Ye, Zezhong
AU  - Prabhu, Sanjay
AU  - Tjong, Michael C.
AU  - Zha, Yining
AU  - Zapaishchykova, Anna
AU  - Vajapeyam, Sridhar
AU  - Hayat, Hasaan
AU  - Chopra, Rishi
AU  - Liu, Kevin X.
AU  - Nabavidazeh, Ali
AU  - Resnick, Adam
AU  - Mueller, Sabine
AU  - Haas-Kogan, Daphne
AU  - Aerts, Hugo J.W.L.
AU  - Poussaint, Tina
AU  - Kann, Benjamin H.
TI  - Expert-level pediatric brain tumor segmentation in a limited data scenario with stepwise transfer learning
AID  - 10.1101/2023.06.29.23292048
DP  - 2023 Jan 01
TA  - medRxiv
PG  - 2023.06.29.23292048
4099  - http://medrxiv.org/content/early/2023/09/18/2023.06.29.23292048.short
4100  - http://medrxiv.org/content/early/2023/09/18/2023.06.29.23292048.full
AB  - Purpose Artificial intelligence (AI)-automated tumor delineation for pediatric gliomas would enable real-time volumetric evaluation to support diagnosis, treatment response assessment, and clinical decision-making. Auto-segmentation algorithms for pediatric tumors are rare, due to limited data availability, and algorithms have yet to demonstrate clinical translation.Methods We leveraged two datasets from a national brain tumor consortium (n=184) and a pediatric cancer center (n=100) to develop, externally validate, and clinically benchmark deep learning neural networks for pediatric low-grade glioma (pLGG) segmentation using a novel in-domain, stepwise transfer learning approach. The best model [via Dice similarity coefficient (DSC)] was externally validated and subject to randomized, blinded evaluation by three expert clinicians wherein clinicians assessed clinical acceptability of expert- and AI-generated segmentations via 10-point Likert scales and Turing tests.Results The best AI model utilized in-domain, stepwise transfer learning (median DSC: 0.877 [IQR 0.715-0.914]) versus baseline model (median DSC 0.812 [IQR 0.559-0.888]; p<0.05). On external testing (n=60), the AI model yielded accuracy comparable to inter-expert agreement (median DSC: 0.834 [IQR 0.726-0.901] vs. 0.861 [IQR 0.795-0.905], p=0.13). On clinical benchmarking (n=100 scans, 300 segmentations from 3 experts), the experts rated the AI model higher on average compared to other experts (median Likert rating: 9 [IQR 7-9]) vs. 7 [IQR 7-9], p<0.05 for each). Additionally, the AI segmentations had significantly higher (p<0.05) overall acceptability compared to experts on average (80.2% vs. 65.4%). Experts correctly predicted the origins of AI segmentations in an average of 26.0% of cases.Conclusions Stepwise transfer learning enabled expert-level, automated pediatric brain tumor auto-segmentation and volumetric measurement with a high level of clinical acceptability. This approach may enable development and translation of AI imaging segmentation algorithms in limited data scenarios.Summary Authors proposed and utilized a novel stepwise transfer learning approach to develop and externally validate a deep learning auto-segmentation model for pediatric low-grade glioma whose performance and clinical acceptability were on par with pediatric neuroradiologists and radiation oncologists.Key PointsThere are limited imaging data available to train deep learning tumor segmentation for pediatric brain tumors, and adult-centric models generalize poorly in the pediatric setting.Stepwise transfer learning demonstrated gains in deep learning segmentation performance (Dice score: 0.877 [IQR 0.715-0.914]) compared to other methodologies and yielded segmentation accuracy comparable to human experts on external validation.On blinded clinical acceptability testing, the model received higher average Likert score rating and clinical acceptability compared to other experts (Transfer-Encoder model vs. average expert: 80.2% vs. 65.4%)Turing tests showed uniformly low ability of experts’ ability to correctly identify the origins of Transfer-Encoder model segmentations as AI-generated versus human-generated (mean accuracy: 26%).Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis study was supported in part by the National Institutes of Health (NIH) (U24CA194354, U01CA190234, U01CA209414, R35CA22052, and K08DE030216), the National Cancer Institute (NCI) Spore grant (2P50CA165962), the European Union European Research Council (866504), the Radiological Society of North America (RSCH2017), the Pediatric Low-Grade Astrocytoma Program at Pediatric Brain Tumor Foundation, and the William M. Wood Foundation.Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The Institutional Review Board (IRB) of the Dana-Farber/Boston Children's/Harvard Cancer Center gave ethical approval for this work.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesBraTS data including raw MRI images may be requested from The Cancer Image Archive (https://www.cancerimagingarchive.net). Although raw MRI imaging data cannot be shared, all measured results to replicate the statistical analysis are shared at the GitHub webpage: https://github.com/BoydAidan/nnUNet_pLGG. Furthermore, we include test samples from a publicly available data set with deep learning and expert reader annotations.AUCarea under the curveCNNconvolutional neural networkDICOMDigital Imaging and Communications in MedicinepLGGpediatric low-grade gliomaDSCdice similarity coefficient.