PT - JOURNAL ARTICLE AU - Schmidt, Linea AU - Ibing, Susanne AU - Borchert, Florian AU - Hugo, Julian AU - Marshall, Allison AU - Peraza, Jellyana AU - Cho, Judy H. AU - Böttinger, Erwin P. AU - Ungaro, Ryan C. TI - Extraction of Crohn’s Disease Clinical Phenotypes from Clinical Text Using Natural Language Processing AID - 10.1101/2023.10.16.23297099 DP - 2023 Jan 01 TA - medRxiv PG - 2023.10.16.23297099 4099 - http://medrxiv.org/content/early/2023/10/16/2023.10.16.23297099.short 4100 - http://medrxiv.org/content/early/2023/10/16/2023.10.16.23297099.full AB - Background Crohn’s Disease (CD) patient heterogeneity in clinical practice is captured by the Montreal Classification. While the underlying concepts, disease behavior and age at diagnosis, are relevant outcomes and covariates in studies from real-world data, extracting this clinical information through manual chart review is labor-intensive and with limited scalability.Methods We developed and evaluated automated phenotyping algorithms to extract disease behavior and age at diagnosis from clinical narrative texts, using a rule-based approach based on the spaCy framework, and an approach based on zero-shot inference. The underlying data included 49,572 clinical notes and 2,204 radiology reports from 584 CD patients of the Mount Sinai Crohn’s and Colitis Registry. A test set of 200 clinical texts per classification category was labeled at sentence-level, in addition to patient-level ground truth data. The algorithms were evaluated based on their recall, precision, specificity values, and F1-scores.Results For the labeled dataset, an overall Cohen’s kappa inter-annotator agreement of 0.84 was achieved. The rule-based approach yielded high recall and precision values (0.75 - 1.00) on a note level for the behavioral disease phenotype using clinical notes, with slightly reduced performance using radiology reports. For age at diagnosis, recall and precision values of 0.81 and 0.88 were achieved on note-level, respectively. For both categories, the performance on patient-compared to note-level was reduced, potentially due to the accumulation of false positives and limitations in the data availability.Conclusion Based on our newly annotated dataset, we demonstrated the feasibility of automatically extracting disease behavior and age at diagnosis from clinical text. The resulting labels may facilitate extensive cohort analyses based on electronic health records, and support chart review processes in the future.Competing Interest StatementThe authors have declared no competing interest.Funding StatementThis work is supported in part through the use of the research platform AI-Ready Mount Sinai (AIR.MS), and through the MSDW resources and staff expertise provided by Scientific Computing and Data at the Icahn School of Medicine at Mount Sinai. This work was funded by the Joachim-Herz Foundation (to SI), and the Hasso Plattner Foundation (HPF).Author DeclarationsI confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.YesThe details of the IRB/oversight body that provided approval or exemption for the research described are given below:The study was approved by the Program of Protection for Human Subjects at the Icahn School of Medicine at Mount Sinai.I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.YesI understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).YesI have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.YesThe data are not publicly available as it contains protected health information.