CONORM: Context-Aware Entity Normalization for Adverse Drug Event Detection

Abstract

Adverse drug events (ADEs) are a critical aspect of patient safety and pharmacovigilance, with significant implications for patient outcomes and public health monitoring. The increasing availability of electronic health records, social media, and online patient forums provides valuable yet challenging unstructured data sources for ADE surveillance. To address these challenges, we introduce CONORM, a novel framework that integrates named entity recognition (NER) and entity normalization (EN) for ADE extraction across diverse textual domains. CONORM comprises two components: CONORM-NER, a subword-aware, transformer-based model for ADE detection, and CONORM-EN, an entity normalization module using dual-encoder embeddings and dynamic context refining. CONORM was evaluated on SMM4H 2023 (tweets), CADEC (forum posts), and TAC (product labels), where it consistently outperformed existing methods. It achieved end-to-end F1-scores of 63.86% on SMM4H, 72.45% on CADEC, and 84.99% on TAC, surpassing existing solutions by an average margin of 35%. The results demonstrate CONORM’s adaptability across informal, semi-formal, and formal text sources. Stratified analyses showed stronger results for well-represented system organ classes (SOCs) and moderate-to-long contexts. CONORM offers a scalable and reproducible solution for real-world pharmacovigilance, with pre-computed target embeddings enhancing inference efficiency for large medical ontologies. Its generalization capabilities across diverse text domains establish it as a robust tool for ADE surveillance. To support reproducibility, all source code is publicly available at https://github.com/ds4dh/CONORM. Future work will explore multilingual capabilities, rare SOC data augmentation, and optimizations for long-form narratives.