RT Journal Article SR Electronic T1 REM: An Integrative Rule Extraction Methodology for Explainable Data Analysis in Healthcare JF medRxiv FD Cold Spring Harbor Laboratory Press SP 2021.01.25.21250459 DO 10.1101/2021.01.25.21250459 A1 Shams, Zohreh A1 Dimanov, Botty A1 Kola, Sumaiyah A1 Simidjievski, Nikola A1 Terre, Helena Andres A1 Scherer, Paul A1 Matjašec, Urška A1 Abraham, Jean A1 Jamnik, Mateja A1 Liò, Pietro YR 2021 UL http://medrxiv.org/content/early/2021/04/28/2021.01.25.21250459.abstract AB Deep learning models are receiving increasing attention in clinical decision-making, however the lack of explainability impedes their deployment in day-to-day clinical practice. We propose REM, an explainable methodology for extracting rules from deep neural networks and combining them with rules from non-deep learning models. This allows integrating machine learning and reasoning for investigating basic and applied biological research questions. We evaluate the utility of REM in two case studies for the predictive tasks of classifying histological and immunohistochemical breast cancer subtypes from genotype and phenotype data. We demonstrate that REM efficiently extracts accurate, comprehensible rulesets from deep neural networks that can be readily integrated with rulesets obtained from tree-based approaches. REM provides explanation facilities for predictions and enables the clinicians to validate and calibrate the extracted rulesets with their domain knowledge. With these functionalities, REM caters for a novel and direct human-in-the-loop approach in clinical decision-making.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis work was supported by The Mark Foundation Institute for Integrated Cancer Medicine (MFICM). MFICM is hosted at the University of Cambridge, with funding from The Mark Foundation for Cancer Research (NY, U. S. A.) and the Cancer Research UK Cambridge Centre [C9685/A25177] (UK).Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:N/AAll necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.YesThe data used is publicly available.MLMachine LearningMRMachine ReasoningDNNDeep Neural NetworkREMRule Extraction MethodologyREM-DRule Extraction Methodology from Deep Neural NetworksREM-TRule Extraction Methodology from Trees