Histopathological evaluation of abdominal aortic aneurysms with deep learning ============================================================================= * Fiona R. Kolbinger * Omar S. M. El Nahhas * Maja Carina Nackenhorst * Christine Brostjan * Wolf Eilenberg * Albert Busch * Jakob Nikolas Kather ## Abstract Computational analysis of histopathological specimens holds promise in identifying biomarkers, elucidating disease mechanisms, and streamlining clinical diagnosis. However, the application of deep learning techniques in vascular pathology remains underexplored. Here, we present a comprehensive evaluation of deep learning-based approaches to analyze digital whole-slide images of abdominal aortic aneurysm samples from 369 patients from three European centers. Deep learning demonstrated robust performance in predicting inflammatory characteristics, particularly in the adventitia, as well as fibrosis grade and remaining elastic fibers in the tunica media. Overall, this study represents the first comprehensive evaluation of computational pathology in vascular disease and has the potential to contribute to improved understanding of abdominal aortic aneurysm pathophysiology and personalization of treatment strategies, particularly when integrated with radiological phenotypes and clinical outcomes. Keywords * Abdominal aortic aneurysm * computational pathology * vascular pathology * deep learning ## Introduction Computational analysis of histopathological imaging has the potential to identify biomarkers, provide insights into disease patterns and pathophysiological processes, and automate histopathological assessment in clinical routine care. The potential of deep learning (DL)-based approaches in vascular pathology has not been characterized in large patient cohorts to date. We evaluated the performance of DL to identify a range of pathological characteristics including inflammation, fibrosis, elastic fiber degradation, angiogenesis, and calcification from digital whole-slide images (WSI) in an expert-annotated dataset of abdominal aortic aneurysm (AAA) wall samples from 369 patients treated at three European centers. ## Methods ### Patient Cohort Data from a total of 369 patients (84.6% male, mean age 69.1 ± 8.0 years, average maximum diameter 62.9 ± 15.7 mm) undergoing open AAA repair at the Technical University Munich (TUM, n = 287), the University Hospital Würzburg (UHW, n = 36) and the Medical University Vienna (MUV, n = 46), between 2005 and 2019, were included in this study (**Figure 1A**). ![Figure 1:](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/04/24/2024.04.23.24306178/F1.medium.gif) [Figure 1:](http://medrxiv.org/content/early/2024/04/24/2024.04.23.24306178/F1) Figure 1: Histopathological evaluation of abdominal aortic aneurysms with deep learning. **(A)** Hematoxylin and Eosin (HE)-stained whole slide images from 369 patients undergoing open aortic aneurysm repair at three European centers were used to develop weakly-supervised deep learning models classifying pathological variables including inflammation **(B)** and fibrosis in the adventitia **(C)**, as well as remaining elastic fibers in the tunica media **(D)**. Model performance was qualitatively evaluated with heatmaps indicating predicted model scores for each 256 μm x 256 μm patch of the whole slide image, as well as with the most predictive patches from the respective slide. Quantitatively, model performance was evaluated using the area under the receiver-operating characteristic curve (AUC) **(B, C, D)**. Adventitial inflammation **(B)** was classified on a four-category ordinal scale (none, minor, intermediate, major inflammation); the figure displays results from a model differentiating none from the other three classes. Adventitial fibrosis **(C)** was classified on a three-category ordinal scale (minor, intermediate, major fibrosis); the figure displays results from a model differentiating major fibrosis from the other two classes. Remaining elastic fibers in the tunica media **(B)** was classified on a binary scale (< 25%, > 25%); the figure displays results from a model differentiating both classes. Abbreviations: Area under the receiver-operating characteristic curve (AUC), Elastica van Gieson (EvG), hematoxylin and eosin (HE), lumen (L), Medical University Vienna (MUV), Technical University Munich (TUM), University Hospital Würzburg (UHW). Aneurysm samples from the left anterior wall were independently analyzed by three pathologists as described previously1 to create ground truth data for model training and testing. In brief, the following histopathological parameters were evaluated: Grade of inflammation in tunica media [none, minor, intermediate, major], grade of inflammation in adventitia [none, minor, intermediate, major], type of inflammation in adventitia and tunica media [none, acute, chronic], Histological Inflammation Scale of Aneurysms (HISA) grade2 [0, 1, 2, 3, 4], angiogenesis in tunica media [present, not present], calcification in tunica media [present, not present], grade of fibrosis in adventitia [minor, intermediate, major], remaining elastic fibers in tunica media [< 25%, > 25%]. In case of disagreement, consensus was reached through discussion. Hematoxylin and Eosin (HE)- and Elastica van Gieson (EvG)-stained slides were digitized using an Aperio AT2 (Leica, Wetzlar, Germany) slide scanner. Patient sex and smoking history were gathered as binary clinical parameters. ### Computational models Weakly-supervised attention-based multiple instance learning models were trained to identify pathological characteristics from HE- and EvG-stained WSI. All models were trained and internally validated by 5-fold cross validation on the TUM cohort and separately tested on external cohorts from MUV and UHW. The STAMP protocol was utilized to process the WSIs3. In brief, WSI were preprocessed by tessellation into 224 × 224 pixel patches at a magnification of 256 μm per pixel, followed by computational background rejection, and color normalization of HE-stained slides. EvG-stained slides were not color-normalized as no color normalization protocols exist for this staining. Subsequently, features were extracted using a pre-trained histology image encoder. To aggregate patches for slide-level predictions, downstream classification tasks were modeled through weakly-supervised attention-based multiple instance learning4, enabling the correlation of slide-level labels with tissue morphology (**Figure 1A**). Model performance was quantified using the mean area under the receiver operating characteristic (AUC) and standard deviation across 5 folds. High-resolution attention heatmaps for qualitative evaluation were generated through a convolutional equivalent of the attMIL model4. ## Results Based on a multicentric expert-labeled histopathological dataset of resected AAA from 369 patients, we trained DL models to identify histopathological parameters directly from WSI. With regard to inflammatory characteristics, DL could reliably predict presence (binary classification, AUCMUV = 0.75 ± 0.10, AUCUHW = 0.85 ± 0.08) and grade of inflammation (multi-class classification, AUCMUV = 0.73 ± 0.10, AUCUHW = 0.78 ± 0.10) in the adventitia from HE-stained slides (**Figure 1B**). While the models did not include any regional information, areas that were most relevant for prediction of inflammation in the adventitia were indeed localized in the adventitial region of specimens and displayed inflammatory cell infiltration from various cell types (i.e. plasma cells, lymphocytes, **Figure 1B**). In contrast, prediction models for presence and grade of inflammation in the tunica media as well as inflammation type in the adventitia and tunica media yielded random prediction results (AUC overlapping 0.50). With regard to other pathological characteristics, DL could predict the grade of fibrosis in the adventitia (major vs. other, AUCMUV = 0.83 ± 0.02, AUCUHW = 0.72 ± 0.08, **Figure 1C**) and remaining elastic fibers in the tunica media (AUCMUV = 0.97 ± 0.02, AUCUHW = 0.75 ± 0.12, **Figure 1D**). Furthermore, DL could differentiate HISA2 classes (AUCMUV = 0.79 ± 0.10, AUCUHW = 0.77 ± 0.06), historically classified based on the extent of mild and chronic inflammation, lymphocytic infiltration, and fibrosis. Angiogenesis, calcification, sex, and smoking history could not be predicted from HE-stained WSI (AUC overlapping 0.50). Models trained on EvG-stained WSI overall performed similar to models trained on HE-stained WSI for detection of calcification, fibrosis, HISA grade, sex, and smoking history. For prediction of the abovementioned inflammatory parameters, HE-trained models performed considerably superior to EvG-trained models. ## Discussion This study presents the largest evaluation of computational pathology in the context of vascular disease to date. Our results provide evidence that DL can reliably predict histopathological features like inflammation, elastic fiber degradation, and fibrosis from HE-stained AAA pathology slides. In addition, DL can predict the HISA grade from unlabeled pathology, affirming the relevance of this pathological classification established in 19942 and the reproducibility of inflammatory characteristics in histologic examination of AAA. Prior works in computational vascular histopathology have evaluated basic segmentation tasks in smaller, monocentric patient cohorts5,6. Previous DL-based studies in AAA have predominantly focused on aneurysm detection and segmentation or prediction of clinical endpoints such as aneurysm growth or rupture risk from radiological or biomechanical data.7,8 Given the nature of AAA and the impracticability of aortic sampling, our findings cannot provide directly actionable clinical insights. However, our results have the potential to contribute to improved understanding of AAA biology and treatment, especially when set in relation to other data modalities (i.e., radiological imaging, clinical risk factors) in multimodal prediction models. Our finding of computationally discernible inflammatory patterns in the adventitia implies potential for non-invasive diagnostic approaches: Imaging techniques like PET, SPECT, and radiolabeled white blood cell scintigraphy can non-invasively assess inflammation levels in the adventitia9 and could potentially identify patients who could benefit from pharmacotherapy targeting inflammation10. To maximize translational impact, our work warrants linking of AAA histopathological phenotypes with respective radiological and biomechanical phenotypes as well as clinical outcomes. ## Declarations ### Abbreviations AAA : Abdominal Aortic Aneurysm AUC : Area under the receiver operating characteristic EvG : Elastica van Gieson HE : Hematoxylin and Eosin HISA : Histological Inflammation Scale of Aneurysms MUV : Medical University Vienna TUM : Technical University Munich UHW : University Hospital Würzburg WSI : Whole-slide image ## Disclosure of Interest OSMEN holds shares in StratifAI GmbH. JNK declares consulting services for Owkin, France, DoMore Diagnostics, Norway, Panakeia, UK, Scailyte, Switzerland, Cancilico, Germany, Mindpeak, Germany, MultiplexDx, Slovakia, and Histofy, UK; furthermore he holds shares in StratifAI GmbH, Germany, has received a research grant by GSK, and has received honoraria by AstraZeneca, Bayer, Eisai, Janssen, MSD, BMS, Roche, Pfizer and Fresenius. The mentioned competing interests are related to the computational analysis of histopathology slides, which is the main topic of this research. All other authors declare no conflicts of interest. ## Data Availability Statement Due to data privacy regulations, the raw imaging data cannot be published along with this work. The feature vectors extracted from the histopathological WSI and the corresponding clinical endpoints analyzed in this work are available at [https://doi.org/10.5281/zenodo.10998463](https://doi.org/10.5281/zenodo.10998463). Code for preprocessing is available at [https://github.com/KatherLab/STAMP](https://github.com/KatherLab/STAMP). Code for modeling is available at [https://github.com/KatherLab/marugoto](https://github.com/KatherLab/marugoto). Code for spatial heatmaps and top-attention tiles is available at [https://github.com/KatherLab/highres-WSI-heatmaps/tree/AAA_heatmaps](https://github.com/KatherLab/highres-WSI-heatmaps/tree/AAA_heatmaps). ## Author Contributions FRK: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing; OSMEN: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing; MCN: Data curation, Formal Analysis, Validation, Visualization, Writing – review & editing; CB: Data curation, Formal Analysis, Validation, Visualization, Writing – review & editing; WE: Data curation, Formal Analysis, Validation, Writing – review & editing; AB: Conceptualization, Data curation, Formal Analysis, Funding acquisition, Project administration, Resources, Supervision, Validation, Visualization, Writing – review & editing; JNK: Conceptualization, Funding acquisition, Methodology, Project administration, Resources, Supervision, Writing – review & editing. ## Funding FRK is supported by the German Cancer Research Center (CoBot 2.0), the Joachim Herz Foundation (Add-On Fellowship for Interdisciplinary Life Science) and the German Research Foundation (Deutsche Forschungsgemeinschaft, DFG) as part of Germany’s Excellence Strategy (EXC 2050/1, Project ID 390696704) within the Cluster of Excellence “Centre for Tactile Internet with Human-in-the-Loop” (CeTI) of the Dresden University of Technology. Furthermore, FRK receives support from the Indiana Clinical and Translational Sciences Institute funded, in part, by Grant Number UM1TR004402 from the National Institutes of Health, National Center for Advancing Translational Sciences, Clinical and Translational Sciences Award. OSMEN is partially funded by the German Federal Ministry of Education and Research (BMBF) through grant 1IS23070, Software Campus 3.0 (TU Dresden), as part of the Software Campus project ‘MIRACLE-AI’. Histology work was partly funded by the German Heart Foundation (Deutsche Herzstiftung) with a grant given to AB (F/46/18). JNK is supported by the German Federal Ministry of Health (DEEP LIVER, ZMVI1-2520DAT111), the German Cancer Aid (DECADE, 70115166), the German Federal Ministry of Education and Research (PEARL, 01KD2104C; CAMINO, 01EO2101; SWAG, 01KD2215A; TRANSFORM LIVER, 031L0312A; TANGERINE, 01KT2302 through ERA-NET Transcan), the German Academic Exchange Service (SECAI, 57616814), the German Federal Joint Committee (TransplantKI, 01VSF21048) the European Union’s Horizon Europe and innovation programme (ODELIA, 101057091; GENIAL, 101096312), the European Research Council (ERC; NADIR, 101114631) and the National Institute for Health and Care Research (NIHR, NIHR213331) Leeds Biomedical Research Centre. The views expressed are those of the author(s) and not necessarily those of the NIH, the NHS, the NIHR or the Department of Health and Social Care. This work was funded by the European Union. Views and opinions expressed are however those of the authors only and do not necessarily reflect those of the European Union. Neither the European Union nor the granting authority can be held responsible for them. ## Ethical Approval This study was performed in accordance with the declaration of Helsinki and its later amendments. The local institutional review boards reviewed and approved the original data collection (Technical University Munich: #2799/10 and #576/18S, University Hospital Würzburg: #188/11, Medical University Vienna: #1729/2014). The overall analysis was approved by the ethics committee of the Technical University Dresden (BO-EK-444102022). All patients had a clinical indication for open AAA repair and gave informed consent to data collection and analysis. * Received April 23, 2024. * Revision received April 23, 2024. * Accepted April 24, 2024. * © 2024, Posted by Cold Spring Harbor Laboratory This pre-print is available under a Creative Commons License (Attribution-NonCommercial-NoDerivs 4.0 International), CC BY-NC-ND 4.0, as described at [http://creativecommons.org/licenses/by-nc-nd/4.0/](http://creativecommons.org/licenses/by-nc-nd/4.0/) ## References 1. 1.Busch, A. Abdominal aortic aneurysms harbor different histomorphology not associated with classic risk factors, the HistAAA study. medRxiv 2024.04.16.24305904 (2024) doi:10.1101/2024.04.16.24305904. [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoibWVkcnhpdiI7czo1OiJyZXNpZCI7czoyMToiMjAyNC4wNC4xNi4yNDMwNTkwNHYxIjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjQvMDQvMjQvMjAyNC4wNC4yMy4yNDMwNjE3OC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 2. 2.Rijbroek, A., Moll, F. L., von Dijk, H. A., Meijer, R. & Jansen, J. W. Inflammation of the abdominal aortic aneurysm wall. Eur. J. Vasc. Surg. 8, 41–46 (1994). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/S0950-821X(05)80118-5&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=8307214&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F24%2F2024.04.23.24306178.atom) 3. 3.El Nahhas, O. S. M. et al. From Whole-slide Image to Biomarker Prediction: A Protocol for End-to-End Deep Learning in Computational Pathology. arXiv [cs.CV] (2023). 4. 4.El Nahhas, O. S. M. et al. Regression-based Deep-Learning predicts molecular biomarkers from pathology slides. Nat. Commun. 15, 1–13 (2024). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1038/s41467-024-45766-2&link_type=DOI) 5. 5.Thorsted, B. et al. Artificial intelligence assisted compositional analyses of human abdominal aortic aneurysms ex vivo. Front. Physiol. 13, 840965 (2022). 6. 6.Niemann, A., Talagini, A., Kandapagari, P., Preim, B. & Saalfeld, S. Tissue segmentation in histologic images of intracranial aneurysm wall. Interdisciplinary Neurosurgery 26, 101307 (2021). 7. 7.Lu, J.-T. et al. DeepAAA: clinically applicable and generalizable detection of abdominal aortic aneurysm using deep learning. arXiv [eess.IV] (2019). 8. 8.Kim, S. et al. Deep Learning on Multiphysical Features and Hemodynamic Modeling for Abdominal Aortic Aneurysm Growth Prediction. IEEE Trans. Med. Imaging 42, 196–208 (2023). 9. 9.Husmann, L. et al. Imaging characteristics and diagnostic accuracy of FDG-PET/CT, contrast enhanced CT and combined imaging in patients with suspected mycotic or inflammatory abdominal aortic aneurysms. PLoS One 17, e0272772 (2022). 10. 10.Miyake, T. & Morishita, R. Pharmacological treatment of abdominal aortic aneurysm. Cardiovasc. Res. 83, 436–443 (2009). [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/cvr/cvp155&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=19454489&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F04%2F24%2F2024.04.23.24306178.atom)