PT - JOURNAL ARTICLE AU - Westhaeusser, Fabian AU - Fuhlert, Patrick AU - Dietrich, Esther AU - Lennartz, Maximilian AU - Khatri, Robin AU - Kaiser, Nico AU - Röbeck, Pontus AU - Bülow, Roman AU - von Stillfried, Saskia AU - Witte, Anja AU - Ladjevardi, Sam AU - Drotte, Anders AU - Severgardh, Peter AU - Baumbach, Jan AU - Puelles, Victor G. AU - Häggman, Michael AU - Brehler, Michael AU - Boor, Peter AU - Walhagen, Peter AU - Dragomir, Anca AU - Busch, Christer AU - Graefen, Markus AU - Bengtsson, Ewert AU - Sauter, Guido AU - Zimmermann, Marina AU - Bonn, Stefan TI - Robust, credible, and interpretable AI-based histopathological prostate cancer grading AID - 10.1101/2024.07.09.24310082 DP - 2024 Jan 01 TA - medRxiv PG - 2024.07.09.24310082 4099 - http://medrxiv.org/content/early/2024/07/10/2024.07.09.24310082.short 4100 - http://medrxiv.org/content/early/2024/07/10/2024.07.09.24310082.full AB - Background Prostate cancer (PCa) is among the most common cancers in men and its diagnosis requires the histopathological evaluation of biopsies by human experts. While several recent artificial intelligence-based (AI) approaches have reached human expert-level PCa grading, they often display significantly reduced performance on external datasets. This reduced performance can be caused by variations in sample preparation, for instance the staining protocol, section thickness, or scanner used. Another limiting factor of contemporary AI-based PCa grading is the prediction of ISUP grades, which leads to the perpetuation of human annotation errors.Methods We developed the prostate cancer aggressiveness index (PCAI), an AI-based PCa detection and grading framework that is trained on objective patient outcome, rather than subjective ISUP grades. We designed PCAI as a clinical application, containing algorithmic modules that offer robustness to data variation, medical interpretability, and a measure of prediction confidence. To train and evaluate PCAI, we generated a multicentric, retrospective, observational trial consisting of six cohorts with 25,591 patients, 83,864 images, and 5 years of median follow-up from 5 different centers and 3 countries. This includes a high-variance dataset of 8,157 patients and 28,236 images with variations in sample thickness, staining protocol, and scanner, allowing for the systematic evaluation and optimization of model robustness to data variation. The performance of PCAI was assessed on three external test cohorts from two countries, comprising 2,255 patients and 9,437 images.Findings Using our high-variance datasets, we show how differences in sample processing, particularly slide thickness and staining time, significantly reduce the performance of AI-based PCa grading by up to 6.2 percentage points in the concordance index (C-index). We show how a select set of algorithmic improvements, including domain adversarial training, conferred robustness to data variation, interpretability, and a measure of credibility to PCAI. These changes lead to significant prediction improvement across two biopsy cohorts and one TMA cohort, systematically exceeding expert ISUP grading in C-index and AUROC by up to 22 percentage points.Interpretation Data variation poses serious risks for AI-based histopathological PCa grading, even when models are trained on large datasets. Algorithmic improvements for model robustness, interpretability, credibility, and training on high-variance data as well as outcome-based severity prediction gives rise to robust models with above ISUP-level PCa grading performance.Competing Interest StatementFW, PF, AD, PS, PW, CB, EB, and SB work part time for Spearpoint Analytics AB, a company developing AI-based digital pathology solutions. The authors declare no other conflicts of interest.Funding StatementThis work is supported by the Department of Defense Prostate Cancer Research Program, DOD Award No W81XWH-18-2-0013, W81XWH-18-2-0015, W81XWH-18- 2-0016, W81XWH-18-2-0017, W81XWH-18-2-0018 and W81XWH-18-2-0019 PCRP Prostate Cancer Biorepository Network (PCBN) JHU 726: We acknowledge the contributions of Dr. Angelo M. De Marzo and Helen L. Fedor from the Brady Urological Research Institute Prostate Specimen Repository at the Johns Hopkins University School of Medicine and Dr. Elizabeth A. Platz from the Department of Epidemiology at the Johns Hopkins Bloomberg School of Public Health for the generation of the Case PSA Progression TMAs for the prostate cancer recurrence nested case-control study, funded in part by the Prostate SPORE Pathology Core (P50 CA58236), Oncology Tissue and Imaging Services at the Sidney Kimmel Comprehensive Cancer Center (P30 CA006973), and a DOD grant (DAMD 17-03- 0273). JHU 235: We acknowledge Drs. Patrick C. Walsh, Alan Partin and Misop Han of The Johns Hopkins University School of Medicine Brady Urological Research Institute for dataset curation for the 235 patient Natural History of Prostate Cancer TMA. PB is supported by the German Research Foundation (DFG, Project IDs 322900939, 432698239 & 445703531), European Research Council (ERC Consolidator Grant No 101001791), and the Federal Ministry of Education and Research (BMBF, STOP-FSGS-01GM2202C). RK was supported by DFG FOR5068 P9 and the 3R initiative of the UKE, PF and MB by DFG SFB1286 SP02, and SB by EU E-rare MAXOMOD, the M3I excellence initiative of the UKE, and DFG SFB1192 B8 and C3. NK was supported by DFG SFB1192 B8 and AW, MZ, and JB by CDL FLIGHT of the University of Hamburg. ED was supported by DFG KFO306 and FW by DFG KFO296. Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:UKE: The use of archived remnants of diagnostic tissues for manufacturing of TMAs and their analysis for research purposes as well as patient data analysis has been approved by local laws (HmbKHG, paragraph 12) and by the Ethics commission Hamburg (WF-049/09). All work has been carried out in compliance with the Helsinki Declaration. NYU, JHU: Both cohorts were accessed via the Prostate Cancer Biorepository Network (PCBN), available at https://prostatebiorepository.org/. The Johns Hopkins University School of Medicine Institutional Review Board and The New York University School of Medicine Institutional Review Board provided ethical regulatory approval for collection and disbursement of data and materials from the respective institutions. The need for acquiring informed consent was waived by the respective institutional ethical review boards. MMX: For ethical approval of the data refer to Saemundsen et al. (Validation of the prognostic value of a three-gene signature and clinical parameters-based risk score in prostate cancer patients) at https://doi.org/10.1002/pros.24530. UPP: The anonymized SPROB20 dataset is publicly available at https://datahub.aida.scilifelab.se/10.23698/aida/sprob20. In our work we have used patient outcome information which is not part of the public dataset. The ethical approval is held by Dr. Michael Haeggman with ID number 2020-00843 relating to the project Fusionsledd biopsi av prostatacancer infoer eventuell kurativ behandling; Tumoergradering med bildanalysbaserad teknik samt molekylaer prognostik av prostatacancer med Prostatype. I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesData from Prostate Cancer Biorepository Network (PCBN), namely the JHU and NYU datasets in this work are available upon request at https://prostatebiorepository.org/.The PANDA dataset with corresponding GT segmentation masks is available on the challenge website at https://panda.grand-challenge.org/data/.The UPP dataset images are publicly available under the name SPROB20 at https://datahub.aida.scilifelab.se/10.23698/aida/sprob20. However, the public version is anonymized and does not provide metadata such as endpoint information for the individual biopsies and patients.The UKEhv, UKE-sealed, and MMX datasets are not publicly available.