Generative AI and Large Language Models in Reducing Medication Related Harm and Adverse Drug Events – A Scoping Review

Jasmine Chiat Ling Ong, Chen Michael, Ning Ng, Kabilan Elangovan, Nichole Yue Ting Tan, Liyuan Jin, Qihuang Xie, Daniel Shu Wei Ting, Rosa Rodriguez-Monguio, David W. Bates, Nan Liu
doi: https://doi.org/10.1101/2024.09.13.24313606

Abstract

Background Medication-related harm has a significant impact on global healthcare costs and patient outcomes, accounting for deaths in 4.3 per 1000 patients. Generative artificial intelligence (GenAI) has emerged as a promising tool in mitigating risks of medication-related harm. In particular, large language models (LLMs) and well-developed generative adversarial networks (GANs) showing promise for healthcare related tasks. This review aims to explore the scope and effectiveness of generative AI in reducing medication-related harm, identifying existing development and challenges in research.

Methods We searched for peer reviewed articles in PubMed, Web of Science, Embase, and Scopus for literature published from January 2012 to February 2024. We included studies focusing on the development or application of generative AI in mitigating risk for medication-related harm during the entire medication use process. We excluded studies using traditional AI methods only, those unrelated to healthcare settings, or concerning non-prescribed medication uses such as supplements. Extracted variables included study characteristics, AI model specifics and performance, application settings, and any patient outcome evaluated.

Findings A total of 2203 articles were identified, and 14 met the criteria for inclusion into final review. We found that generative AI and large language models were used in a few key applications: drug-drug interaction identification and prediction; clinical decision support and pharmacovigilance. While the performance and utility of these models varied, they generally showed promise in areas like early identification and classification of adverse drug events and support in decision-making for medication management. However, no studies tested these models prospectively, suggesting a need for further investigation into the integration and real-world application of generative AI tools to improve patient safety and healthcare outcomes effectively.

Interpretation Generative AI shows promise in mitigating medication-related harms, but there are gaps in research rigor and ethical considerations. Future research should focus on creation of high-quality, task-specific benchmarking datasets for medication safety and real-world implementation outcomes.

Introduction

Medication-related harm poses significant health and economic burden globally. The global prevalence of medication related harm was 12%, of which 15% was severe and fatal, causing a mortality rate of up 4.3 per 1000 patients.1 In contrast, cardiovascular deaths caused by ischemic heart disease accounted for 1.09 deaths per 1000 patients.2 The economic burden of medication-related harm is estimated at $30.1 billion and 79 billion euros in United States and Europe respectively.3 Medication-related harm, also termed as adverse drug events (ADEs) include preventable or non-preventable harm caused by interventions related to medication use.4 Preventable medication error can occur at any step from the physician prescribing medications to the patient receiving the medication. In turn, ADEs are under active surveillance during healthcare delivery to patients with health systems or pharmacovigilance activities.

Advances in artificial intelligence (AI), digitization of health records, and accessibility to electronic patient records have been shown to reduce the occurrence, duration, and severity of ADEs.5–7 An AI-powered system has been reported to reduce inpatient prescribing error by up to 20%.8 However, traditional predictive models are still limited by the lack of in-depth clinical reasoning, poor interoperability in electronic health record systems (EHRs), difficulty in detecting rare events or interactions, and paucity of models that leverage unstructured data. Based on existing system, overlooked ADEs would lead to significant healthcare complications while trivial or clinical insignificant effects are over emphasized leading to healthcare administrative burden. Thus, with significant promises to address such unbalanced issue, generative AI (GenAI) and large language models (LLMs) may enable novel approaches previously unfeasible with conventional methods. For instance, preliminary studies have explored the potential of ChatGPT to recognize adverse drug reactions9, pharmacovigilance signal detection10, and automated medication chart review.11,12

This systematic review summarizes the breadth and depth of existing literature on how generative AI have been utilized to reduce ADEs and highlights areas for future investigation.

Methods

Search Strategy and Selection Criteria

This systematic review was conducted according to PRISMA guidelines.16 We searched PubMed, Web of Science, Embase, and Scopus to identify studies published between 1st January 2012 to 18th February 2024, related to application of generative AI in reducing medication related harm. Details of the search terms are provided in [Supplement III].

Studies were included if they were published in English, described the development or application of generative AI in mitigating potential medication related harms in the care delivery process (Figure 1), and were peer-reviewed original research, review and viewpoints, structured reviews of the literature reported in accordance with PRISMA guidelines, conference abstracts, case reports. We excluded studies that utilized solely predictive modeling approaches or investigated ADEs related to dietary supplements or use of medication not prescribed for the individual.

Figure 1.
Figure 1.

Archetypical care delivery process and potential points of error.

Based on these criteria, abstracts were screened for eligibility by two independent reviewers using a standardized tool. If no exclusion criteria were apparent in the abstract, it was included for manuscript review. Full-text manuscripts were conducted by two independent reviewers. Studies that did not meet the selection criteria were excluded at this stage. In cases of discrepancy between reviewers, eligibility was determined by a third reviewer.

Data Analysis

We used a standardized form to extract pertinent information, including study characteristics, model details, application setting, outcome measures, findings, and reported challenges and limitations [Supplement IV]. We did not perform a critical appraisal of study quality as our primary objective is to characterise the scope of research in this field, identify research trends and gaps. The wide range of study designs and outcomes also precluded the application of a uniform quality assessment criterion.

Role of funding source

There was no funding source for this study.

Results

The search yielded 2203 articles from all databases, with 1734 remaining after removing duplicates. After applying inclusion and exclusion criteria, 14 articles were eligible for this review.

Figure 1a:
Figure 1a:

PRISMA flow diagram17

Study Characteristics

Studies evaluated the performance of GenAI models for various applications, as summarized in Table 2. Four studies focused on the identification, classification, or prediction of drug-drug interactions (DDI).18–21 Three studies assessed the performance and utility of GenAI as decision support tools in benzodiazepine deprescribing22, aid dosing calculation of crushed tablets23 and provision of drug information24. Majority of studies focused on the application of GenAI in adverse event monitoring from specific drug classes and enhancing pharmacovigilance processes. Study designs were predominantly observational and cross-sectional. None of the studies tested models prospectively in their respective settings of application. The proposed applications of GenAI were broadly distributed across clinical (community, inpatient care) and public health settings.

View this table:
Table 1. Key concepts and definitions
View this table:
Table 2: Summary of studies included in review
View this table:
Table 3: Types of generative models and data characteristics

Dataset Characteristics

Pharmacovigilance tasks often used public health databases, including FDA Adverse Event Reporting System (FAERS) Public Dashboard25, Health Canada ADR reporting dashboard26, China Food and Drug Adminstration27. Various datasets were used for named entity recognition tasks in ADE detection, such as user-generated content from web platforms (Medicitalia28, WebMD29) and open or restricted access datasets (CoNLL200330, BioCreative V CDR31, n2c232). Models built for DDI prediction utilized closed-source or in-house datasets such as DrugBank24,33. Testing datasets are often accrued from prior published studies, such as drug-drug interaction and deprescribing case scenarios22,34 with one study using an in-house retrospective cohort of medication prescriptions.19

Model Types

Proprietary LLMs featured frequently in the reviewed studies, including various versions of ChatGPT and Google Bard. Studies used simple prompts or iterative prompting to generate responses on pretrained LLMs. None of the studies reported the use of additional techniques to enhance model performance, such as retrieval augmented generation or fine-tuning.

Custom-developed models adopt iterations of generative adversarial networks (GAN) and variational autoencoders (VAE) in the studies reviewed. The GTCACS35 approach was a three-step approach to better identify discussion topics from social media texts. GAN achieved dimensionality reduction, keyword clustering and summarization. DeepSAVE36, a deep learning framework used an enriched VAE for dimensionality reduction through parsimonious modelling of events captured on social media platform. In one study, BART (Bidirectional and Auto-Regressive Transformers) fine-tuned with a small amount of ADR-specific named entities (few-shot learning) was adapted to allow automated identification of diverse ADEs using small volumes of annotated data.37 GAN was adapted in DGANDDI21 into a graph attention network that encode drug attributes. DGANDDI was capable of binary and multi-class prediction tasks for drug-drug interactions using an enhanced and augmented multi-dimensional dataset generated by GAN. In a similar fashion, GAN was used to generate artificial features to augment data distribution in an imbalanced spontaneous reporting dataset.27

Model Performance

For tasks that assist clinical decision making, reference standards include information from knowledge databases e.g. Lexicomp® and expert opinion (healthcare providers or pharmacologists). Most commonly reported metrics include accuracy, sensitivity, specificity, F-1 statistic, precision, recall, AUC and AUPRC. Bespoke metrics include qualitative assessment of model responses by human experts, graded on Likert scales for quality, completeness or satisfaction. One study used ChatGPT-4 for qualitative evaluation of model performance, performed in parallel with human expert evaluation.38

Drug-Drug Interaction Classification and Prediction18–21

In prediction of potential drug interaction pairs, GAN-based models achieved high accuracy rate. GANs are generative models that learn from the distribution of data or images to create large, realistic synthetic data.39 DGANDDI outperformed baseline methods in both binary and multi-class DDI prediction tasks. In binary prediction, it achieved an accuracy of 96.10%, AUPR of 99.27%, and AUROC of 99.26%. In the multi-class prediction task, DGANDDI attained an accuracy of 95.89%, AUPR of 97.29%, and AUROC of 99.97%.

Proprietary LLMs were used to classify DDIs. One study compared the performance of different LLMs including Microsoft Bing AI, ChatGPT-3.5 and ChatGPT-4 and Google Bard.18 When Micromedex was used as the reference standard, accuracy of LLMs ranged between 0.469 to 0.788, with Microsoft Bing AI demonstrating the best performance. Sensitivity was comparable across all LLMs, but specificity was significantly lower for ChatGPT-3.5 and ChatGPT-4. In another study, the Google Bard was used to screen prescriptions for drug-drug interactions, demonstrated low degree of agreement with predictions from Lexicomp.19 There was a nil to slight agreement between interaction risk rating (κ=0.01), severity rating (κ=0.02), and reliability (κ =-0.02). Conversely, ChatGPT (version not reported) was found to be highly accurate in identifying drug-drug interactions in 39 out of 40 DDI pairs tested. When prompted to explain its answer, ChatGPT produced responses that was highly readable.

Decision Support22–24

In decision support applications, a study leveraging GPT-4 for benzodiazepine deprescribing reported high degree of overall agreement between LLM and human expert in identifying cases eligible for deprescribing.22 Agreement on four different deprescribing criteria was varied, ranging 74.7% to 91.3% (lack of indication: κ = .352, P < .001; prolonged use: κ = .088, P = .280; safety concerns: κ = .123, P = .006; incorrect dosage: κ = .264, P = .001). Qualitative analysis of GPT-4 responses found that up to 22% were ambiguous, generic and contained inconsistencies. Another study introduced a web-based calculator developed to guide dosing calculation, particularly in paediatric care where such errors are prevalent.23 The authors used ChatGPT (version not reported) and Visual Studio to write the underlying HTML code for dose division calculations and webpage interface creation. The webpage’s reliability and feasibility were then assessed using retrospective data and validated questionnaires, scoring 88.38 on the System Usability Scale. Accuracy and reproducibility of the calculator was not evaluated.

In the provision of pharmacovigilance related enquiries, ChatGPT-4 responses was compared against responses by pharmacovigilance specialists. The median score (IQR) of the ChatGPT’s responses on a 10-point Likert scale was 4.8 (3–7.3), with a specific focus on drug causality scoring lower at 3.7 (3– 6.3), and information on medication and proper use scoring slightly better at 5 (3.2–8.3). The authors conclude that chatbot’s responses were generally not acceptable, especially in terms of precision and clinical relevance.

Pharmacovigilance27,35–37,40

For signal detection and ADR classification, studies used generative AI for training data augmentation and dimension reduction. These models e.g. GTCACS outperformed non GenAI methods on internal validity measures. The DeepSAVE model, which uses a VAE approach, was tested on a dataset comprising of 104 million user search queries and 800 events. DeepSAVE outperformed existing methods (e.g. disproportionality analysis, DA atop Event Mention Classifier) with the highest F-measure across all validation datasets. A GAN-based classification model developed to automatically evaluate risk categories of drugs during post-marking surveillance demonstrated highest accuracy of 97.9% when compared against existing models.

In one study, authors demonstrated few-shot learning with LightNER and BART, the ADR recognition performance in low-resource datasets significantly improved. For instance, the LightNER model fine-tuned using the N2C2 dataset, achieved an F1-score of 61.42%, indicating the model’s effective transfer of task knowledge from rich-resource to low-resource settings. GPT-3 was used in another study to generate a comprehensive lexicon of drug abuse synonyms from social media sources. Coupled with automated API queries and simple automated filters (e.g. google filters), the proposed method yielded precision of 0.859 and 0.770, recall of 0.431 and 0.395 for alprazolam and fentanyl respectively.

LLM was harnessed to improve efficiency of pharmacovigilance process in one study, where authors used iterative prompting of GPT-4 to review and summarize food effects on drugs from drug review documents. Final draft summaries generated by GPT-4 were rated by FDA professionals, with 85% rated as factually consistent with reference summaries. This showcases GPT-4’s potential to aid in faster and more reliable drug assessment processes.

Discussion

As healthcare systems increasingly prioritize patient safety, the integration of AI has the potential to enhance the detection and prevention of ADEs, and, by extension, reduce the substantial economic cost. Our scoping review revealed three key applications of GenAI in the literature to date: identification and prediction of drug-drug interactions, provision of decision support in medication management, and automation of pharmacovigilance activities.

Effectiveness of Generative AI in Enhancing Safety

Drug-drug interactions make up nearly 3% of all hospital admissions and account for up to 5% of all inpatient medication errors.41 Harmful DDIs are often only reported from post-marketing surveillance activities, rather than at the clinical trial stage.42 Our review included studies that predict potential DDIs pre-clinically. Performance of models augmented by GAN outperforms those trained using traditionally augmented data using a fraction of the original training dataset.43,44 GAN can be a useful tool in enhancing prediction accuracy where data is limited. On the other hand, LLMs demonstrated variable performance in screening DDIs from prescriptions. Studies frequently used simple prompting strategies to elucidate response from LLMs, with no additional techniques used to provide contextual knowledge or reduce incorrect responses (or “hallucinations”). Methods such as retrieval augmented generation (RAG) or fine-tuning may allow LLMs to tailor responses to specified tasks through provision of contextual knowledge (e.g. drug-drug interaction database).45,46 The advantage of such techniques have been shown in other clinical tasks, including differential diagnosis47, evidenced-based decision support46,48 and patient chart review49. These techniques however, may rely on well-curated, clinically adjudicated drug-drug interaction datasets that are not often freely available.

As decision support tools, studies adopting LLMs are mainly exploratory in nature. We found a wide range of tasks and purposes (e.g. prescription review, dosage calculation, and answering medication enquiry). These broad applications are enabled by generalist properties of large language models.50 LLMs demonstrate capacity to perform tasks with little to no task-specific training, also known as “zero-shot” or “few-shots” learning. In the context of reducing medication harm, LLMs may simulate clinical reasoning and inferential skills across diverse medical disciplines, drug classes and user settings without the need for explicit training. For instance, an LLM trained to screen prescriptions for inappropriate benzodiazepine use may be adapted easily to screen for inappropriate drug use in elderly patients. In addition, LLMs are well poised as medical chatbots, given their text generation capabilities demonstrating high degree of fluency, empathy, and personalization, even outperforming clinicians.51 These explorations, however, highlight existing challenges to clinical adoption of LLMs. While studies to date are in research phase and no exploration in terms of auto-piloting or co-piloting as modes of clinical integration. Accuracy, reliability and consistency of LLM responses using general purpose LLMs such as ChatGPT precludes its autonomous use in clinical settings.

A promising area of LLM application in enhancing efficiency and impact is in ADE monitoring and pharmacovigilance, where GenAI tools may enhance timeliness and accuracy of ADR detection from specific medication classes. Our review has shown that GenAI enhanced accuracy of signal detection, disease and drug entity recognition over conventional natural language processing tools. LLMs are able to handle a wide breadth of data sources (i.e. electronic health records, online databases, and social media platform), facilitating the detection of rare events and offering a generalist capability that is essential for continuous learning and adaptation.52 Automation of specific tasks in pharmacovigilance that is traditionally resource-intensive is a potential avenue for productivity gain with the use of GenAI models.53

Clinical Implications

In our review, we are unable to provide conclusive evidence that GenAI will reduce medication-related harms when applied in clinical settings. Along the continuum of medication use process, GenAI models were actually only adopted for highly selective domains and tasks. While a comprehensive review or metrics of medication-related tasks across multiple domains or tasks across different GenAI models have yet to be concluded. In other words, current generative AI in research are still applied as narrow based AI focused on specific task, yet to be explored or achieve their “generative” potential for medication safety. Non-generative AI models was evaluated in another review, where 78 articles described the application of AI in reducing ADEs.54 A variety of AI techniques were described including neural networks and tree-based algorithms in predicting potential ADEs and enhancing early detection.

Utilizing diverse data sources like genetic information and electronic health records, these AI models aimed to inform clinical decisions on safe prescribing and medication management.

Instead of applying generative AI models in tasks that mandate deterministic outputs, we propose that LLMs can be adopted in ways to reduce cognitive workload for healthcare professionals. Healthcare professionals work with high volumes of multi-modal patient data and are required to pay attention to details, synthesize information and make clinical decisions in real time. High cognitive load pose risk for burnout and medical errors.55 For example, LLMs can be used to analyse and reduce alert burden in electronic medical records, in medication incident analysis, and summarization in a similar fashion to discharge notes generation.12,56 In a study published after we completed literature search, LLMs were used in a co-pilot system to extract key named entities of online submitted prescriptions and assembly into coherent instructions.57 This system was shown to reduce near-miss events and improved the efficiency of pharmacy operations in a large-scale online pharmacy. Finally, LLMs can be leveraged upon as a tool in patient education and engagement thereby enhancing patient access to critical medication related information.58

A critical evaluation of studies included in our review revealed a lack of adherence to reporting guidelines for AI studies. We did not perform a quality review of the studies in view of the scoping nature of this review and diverse hypotheses of studies included. Checklists and reporting guidelines such as the MI-CLAIM for transparent model reporting59, TRIPOD+AI checklist for comprehensive reporting of predictive models60 and DECIDE-AI checklist for early stage clinical evaluation of AI-based decision support tools61 should be adopted in future studies. However, there is still lack of validated reporting tools for LLM-based AI model, though initial efforts have been made to create LLM-specific frameworks.62 Evaluation or discussions on model fairness, bias and other ethical considerations such as data privacy were also found to be lacking in the included studies.

Limitations

Our study has several limitations. The heterogeneity across studies regarding application, GenAI tools used and setting prevented a formal assessment of predictive validity for different AI models. Diversity of training and testing datasets used precludes generalizability of findings across different demographic groups. Patient outcomes were not reported in all studies, limiting any conclusions about the role of GenAI and its impact on patient outcomes. We limited our review to only peer reviewed articles. We acknowledge that the field of generative AI and LLM is rapidly evolving and a large number of studies may still be in the preprint stage or archived.

Blueprint for Future Studies

Future research should focus on developing and benchmarking generative AI models against established healthcare standards to further validate their performance and cost-effectiveness to ensure their safe integration into clinical practice. There is a need to develop expert curated, high quality training datasets with diverse representation from different geographical, ethnic and social groups. Such datasets, when shared, can facilitate and accelerate training of GenAI models adapted for different applications in the medication use process. For example, an expert annotated dataset of incident reports can be used to fine-tune an LLM-based model to predict risk for medication incidents.63 Other areas of high interest include the use of LLMs for real-time monitoring of drug safety and the exploration of GAN the synthetic generation of training data, which can help overcome the limitations posed by rare ADE occurrences.

Conclusion

GenAI and LLMs demonstrate potential in enhancing medication safety and reducing medication-related harm. Published studies reveal potential areas for successful future implementation. However, the current areas that have been addressed are targeted at only some of the key safety issues of medication safety today. Moreover, research rigor and comprehensive ethical evaluation is lacking in the studies to date. Future studies should address gaps in lack of high-quality datasets specific for medication safety tasks. Continuous update of this review is warranted given the burgeoning nature of this field.

Contributors

NL, JCLO and CM conceptualized and designed the systematic review. JCLO, CM, NN, KB, NYTT, LJ and QX performed article screening and data extraction. RR, DWB, DSWT contributed to the first draft of the report with input from NL. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.

Declaration of Interest

We declare no competing interests.

Data Availability

All data produced in the present study are available upon reasonable request to the authors

View this table:
Supplement I Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) Checklist

View this table:
Supplement II Population, Concept and Contexts (PCC) for the scoping review
View this table:
Supplement III Search Strategy
View this table:
Supplement IV Data abstraction variables

References

  1. 1.↵
    WHO Integrated Health Services MWH. Global burden of preventable medication-related harm in health care: a systematic review. Geneva: World Health Organization. 2023;Licence: CC BY-NC-SA 3.0 IGO.
  2. 2.↵
    M V, GA M, JV T, V F, GA R. The Global Burden of Cardiovascular Diseases and Risk: A Compass for Future Health. Journal of the American College of Cardiology. 12/20/2022 2022;80(25)doi:10.1016/j.jacc.2022.11.005
    OpenUrlCrossRef
  3. 3.↵
    H LL, PJ P. Twenty-First Century Global ADR Management: A Need for Clarification, Redesign, and Coordinated Action. Therapeutic innovation & regulatory science. 2023 Jan 2023;57(1)doi:10.1007/s43441-022-00443-8
    OpenUrlCrossRef
  4. 4.↵
    DW B, DJ C, N L, et al. Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group. JAMA. 07/05/1995 1995;274(1)
    OpenUrl
  5. 5.↵
    DW B, D L, A S, et al. The potential of artificial intelligence to improve patient safety: a scoping review. NPJ digital medicine. 03/19/2021 2021;4<otherinfo>(1)</otherinfo>doi:10.1038/s41746-021-00423-6
    OpenUrlCrossRef
  6. 6.
    C G, M B, B M, P R, F M, P T. Accessing Artificial Intelligence for Clinical Decision-Making. Frontiers in digital health. 06/25/2021 2021;3doi:10.3389/fdgth.2021.645232
    OpenUrlCrossRef
  7. 7.↵
    A C, O A. Role of Artificial Intelligence in Patient Safety Outcomes: Systematic Literature Review. JMIR medical informatics. 07/24/2020 2020;8(7)doi:10.2196/18599
    OpenUrlCrossRef
  8. 8.↵
    JC S, Q C, JC D, DM R, KB J, RA M. Evaluation of a Novel System to Enhance Clinicians’ Recognition of Preadmission Adverse Drug Reactions. Applied clinical informatics. 2018 Apr 2018;9(2)doi:10.1055/s-0038-1646963
    OpenUrlCrossRef
  9. 9.↵
    X H, D E, X L, Y Y, J Q, Z L. Evaluating the performance of ChatGPT in clinical pharmacy: A comparative study of ChatGPT and clinical pharmacists. British journal of clinical pharmacology. 2024 Jan 2024;90(1)doi:10.1111/bcp.15896
    OpenUrlCrossRef
  10. 10.↵
    X W, X X, Z L, W T. Bidirectional Encoder Representations from Transformers-like large language models in patient safety and pharmacovigilance: A comprehensive assessment of causal inference implications. Experimental biology and medicine (Maywood, NJ). 2023 Nov 2023;248(21)doi:10.1177/15353702231215895
    OpenUrlCrossRef
  11. 11.↵
    D R, P P, R K, H K, C V, Y W. Effectiveness of ChatGPT in clinical pharmacy and the role of artificial intelligence in medication therapy management. Journal of the American Pharmacists Association : JAPhA. 2024 Mar-Apr 2024;64(2)doi:10.1016/j.japh.2023.11.023
    OpenUrlCrossRef
  12. 12.↵
    Ong JCL, Jin L, Elangovan K, et al. Development and Testing of a Novel Large Language Model-Based Clinical Decision Support Systems for Medication Safety in 12 Clinical Specialties. 2024/01/29 2024;
  13. 13.
    Medication Without Harm - Global Patient Safety Challenge on Medication Safety. Geneva: World Health Organization LCB-N-SI. https://iris.who.int/bitstream/handle/10665/255263/WHO-HIS-SDS-2017.6-eng.pdf?sequence=1
  14. 14.
    DW B, DL B, MB VV, J S, L L. Relationship between medication errors and adverse drug events. Journal of general internal medicine. 1995 Apr 1995;10(4)doi:10.1007/BF02600255
    OpenUrlCrossRefPubMedWeb of Science
  15. 15.
    L H, AC vG. Pharmacovigilance: methods, recent developments and future perspectives. European journal of clinical pharmacology. 2008 Aug 2008;64(8)doi:10.1007/s00228-008-0475-9
    OpenUrlCrossRefPubMedWeb of Science
  16. 16.↵
    MJ P, JE M, PM B, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ (Clinical research ed). 03/29/2021 2021;372doi:10.1136/bmj.n71
    OpenUrlCrossRef
  17. 17.↵
    Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. 2021-03-29 2021;doi:10.1136/bmj.n71
    OpenUrlFREE Full Text
  18. 18.↵
    FY A-A, M Z, L G, R A-F, AN B. Evaluating the Sensitivity, Specificity, and Accuracy of ChatGPT-3.5, ChatGPT-4, Bing AI, and Bard Against Conventional Drug-Drug Interactions Clinical Tools. Drug, healthcare and patient safety. 09/20/2023 2023;15doi:10.2147/DHPS.S425858
    OpenUrlCrossRef
  19. 19.↵
    DM S, SS S, HB A, AM S, MA M. Screening the Drug-Drug Interactions Between Antimicrobials and Other Prescribed Medications Using Google Bard and Lexicomp® Online™ Database. Cureus. 09/09/2023 2023;15(9)doi:10.7759/cureus.44961
    OpenUrlCrossRef
  20. 20.
    A J, N P, S S, S M, JK B, H M. The Capability of ChatGPT in Predicting and Explaining Common Drug-Drug Interactions. Cureus. 03/17/2023 2023;15(3)doi:10.7759/cureus.36272
    OpenUrlCrossRef
  21. 21.↵
    H Y, K L, J S. DGANDDI: Double Generative Adversarial Networks for Drug-Drug Interaction Prediction. IEEE/ACM transactions on computational biology and bioinformatics. 2023 May-Jun 2023;20(3)doi:10.1109/TCBB.2022.3219883
    OpenUrlCrossRef
  22. 22.↵
    I B, D B, M D, et al. Clinical decision-making in benzodiazepine deprescribing by healthcare providers vs. AI-assisted approach. British journal of clinical pharmacology. 2024 Mar 2024;90(3)doi:10.1111/bcp.15963
    OpenUrlCrossRef
  23. 23.↵
    . J S, M R, TM PK, V P, SH C. Dose 4 You: Dose Division Calculator-A Tool to Reduce Calculation Errors. Hospital pharmacy. 2024 Apr 2024;59(2)doi:10.1177/00185787231207757
    OpenUrlCrossRef
  24. 24.↵
    F M, W S, C dC, et al. Will artificial intelligence chatbots replace clinical pharmacologists? An exploratory study in clinical practice. European journal of clinical pharmacology. 2023 Oct 2023;79(10)doi:10.1007/s00228-023-03547-8
    OpenUrlCrossRef
  25. 25.↵
    @US_FDA. FDA Adverse Event Reporting System (FAERS) Public Dashboard | FDA. 2024; Canada H. Adverse Reaction Database - Canada.ca. 2009-09-24 2009;
  26. 26.↵
    J W, G F, Z L, P H, Y Z, W H. Evaluating Drug Risk Using GAN and SMOTE Based on CFDA’s Spontaneous Reporting Data. Journal of healthcare engineering. 08/27/2021 2021;2021doi:10.1155/2021/6033860
    OpenUrlCrossRef
  27. 27.↵
    Consulti medici e specialisti online: più informati, più sani! @medicitalia. https://www.medicitalia.it
  28. 28.↵
    WebMD. Drugs & Medications; Topramax. https://reviewswebmdcom/drugs/drugreview-14494-topamax-oral. 2024;
  29. 29.↵
    Erik F. Tjong Kim Sang FDM. Introduction to the CoNLL-2003 shared task | Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003 - Volume 4. Proceedings of the seventh conference on Natural language learning at HLT-NAACL. 2003;4 May:142–147. doi:10.3115/1119176.1119195
    OpenUrlCrossRef
  30. 30.↵
    J L, Y S, RJ J, et al. BioCreative V CDR task corpus: a resource for chemical disease relation extraction. Database : the journal of biological databases and curation. 05/09/2016 2016;2016doi:10.1093/database/baw068
    OpenUrlCrossRef
  31. 31.↵
    S H, Y W, F S, O U. The 2019 National Natural language processing (NLP) Clinical Challenges (n2c2)/Open Health NLP (OHNLP) shared task on clinical concept normalization for clinical records. Journal of the American Medical Informatics Association : JAMIA. 10/01/2020 2020;27(10)doi:10.1093/jamia/ocaa106
    OpenUrlCrossRef
  32. 32.↵
    Online D. DrugBank Online | Database for Drug and Drug Target Info. 2024;
  33. 33.↵
    AS B, MA S, A A, EA G, J B, D F. Prevalence and Determinants of Multimorbidity, Polypharmacy, and Potentially Inappropriate Medication Use in the Older Outpatients: Findings from EuroAgeism H2020 ESR7 Project in Ethiopia. Pharmaceuticals (Basel, Switzerland). 08/25/2021 2021;14(9)doi:10.3390/ph14090844
    OpenUrlCrossRef
  34. 34.↵
    GraniGiorgio, LenziAndrea, VelardiPaola. Supporting Personalized Health Care With Social Media Analytics: An Application to Hypothyroidism. research-article. 2021-10-15 2021;doi:4
  35. 35.↵
    F. Ahmad AA, B. Kitchens, D. Adjeroh and D. Zeng. Deep Learning for Adverse Event Detection From Web Search. IEEE Transactions on Knowledge and Data Engineering. 2022;34(6):2681–2695.
    OpenUrl
  36. 36.↵
    Chang C-H, National Sun Yat-sen University DoIM, Kaohsiung,Taiwan, Chang F-Y, Hwang S- Y, Yang CC, Drexel University CoCaI, Philadelphia,USA. Prompting for Few-shot Adverse Drug Reaction Recognition from Online Reviews. IEEE Computer Society; 2023:168-175.
  37. 37.↵
    Y S, P R, J W, et al. Leveraging GPT-4 for food effect summarization to enhance product-specific guidance development via iterative prompting. Journal of biomedical informatics. 2023 Dec 2023;148doi:10.1016/j.jbi.2023.104533
    OpenUrlCrossRef
  38. 38.↵
    A A, A A. Generative adversarial networks and synthetic patient data: current challenges and future perspectives. Future healthcare journal. 2022 Jul 2022;9(2)doi:10.7861/fhj.2022-0013
    OpenUrlAbstract/FREE Full Text
  39. 39.↵
    KA C, RB A. Using GPT-3 to Build a Lexicon of Drugs of Abuse Synonyms for Social Media Pharmacovigilance. Biomolecules. 02/18/2023 2023;13(2)doi:10.3390/biom13020387
    OpenUrlCrossRef
  40. 40.
    LL L, DW B, DJ C, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 07/05/1995 1995;274(1)
    OpenUrl
  41. 41.↵
    Lu Y, Shen D, Pietsch M, et al. A novel algorithm for analyzing drug-drug interactions from MEDLINE literature. OriginalPaper. Scientific Reports. 2015-11-27 2015;5(1):1-10. doi:doi:10.1038/srep17357
    OpenUrlCrossRef
  42. 42.↵
    AA V, FF G, YJ K, et al. Deploying deep learning models on unseen medical imaging using adversarial domain adaptation. PloS one. 10/14/2022 2022;17(10)doi:10.1371/journal.pone.0273262
    OpenUrlCrossRef
  43. 43.↵
    AJ SK, RS C, JG C, et al. Evaluation of Generative Adversarial Networks for High-Resolution Synthetic Image Generation of Circumpapillary Optical Coherence Tomography Images for Glaucoma. JAMA ophthalmology. 10/01/2022 2022;140(10)doi:10.1001/jamaophthalmol.2022.3375
    OpenUrlCrossRef
  44. 44.↵
    C Z, R S, A C, et al. Almanac - Retrieval-Augmented Language Models for Clinical Medicine. NEJM AI. 2024 Feb 2024;1(2)doi:10.1056/aioa2300068
    OpenUrlCrossRef
  45. 45.↵
    Ke Y, Jin L, Elangovan K, et al. Development and Testing of Retrieval Augmented Generation in Large Language Models -- A Case Study Report. 2024/01/29 2024;
  46. 46.↵
    S R, A R, J N, et al. A retrieval-augmented chatbot based on GPT-4 provides appropriate differential diagnosis in gastrointestinal radiology: a proof of concept study. European radiology experimental. 05/17/2024 2024;8(1)doi:10.1186/s41747-024-00457-x
    OpenUrlCrossRef
  47. 47.↵
    S K, M G, M A, A A, LS C, DL S. Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework. NPJ digital medicine. 04/23/2024 2024;7(1)doi:10.1038/s41746-024-01091-y
    OpenUrlCrossRef
  48. 48.↵
    Vaid Aea. Using fine-tuned large language models to parse clinical notes in musculoskeletal pain disorders. The Lancet Digital Health. 2024;5(12):e855–e858.
    OpenUrl
  49. 49.↵
    Tu T, Azizi S, Driess D, et al. Towards Generalist Biomedical AI. research-article. 2024-02-22 2024;doi:10.1056/AIoa2300138
    OpenUrlCrossRef
  50. 50.↵
    IA B, YV Z, D G, et al. Comparison of Ophthalmologist and Large Language Model Chatbot Responses to Online Patient Eye Care Questions. JAMA network open. 08/01/2023 2023;6(8)doi:10.1001/jamanetworkopen.2023.30320
    OpenUrlCrossRef
  51. 51.↵
    M M, O B, ZSH A, et al. Foundation models for generalist medical artificial intelligence. Nature. 2023 Apr 2023;616(7956)doi:10.1038/s41586-023-05881-4
    OpenUrlCrossRefPubMed
  52. 52.↵
    M S, J P, P Y, et al. The Use of Artificial Intelligence in Pharmacovigilance: A Systematic Review of the Literature. Pharmaceutical medicine. 2022 Oct 2022;36(5)doi:10.1007/s40290-022-00441-z
    OpenUrlCrossRef
  53. 53.↵
    A S, W S, MG A, et al. Key use cases for artificial intelligence to reduce the frequency of adverse drug events: a scoping review. The Lancet Digital health. 2022 Feb 2022;4(2)doi:10.1016/S2589-7500(21)00229-6
    OpenUrlCrossRef
  54. 54.↵
    DE E, SN G, S N, et al. Evaluating and reducing cognitive load should be a priority for machine learning in healthcare. Nature medicine. 2022 Jul 2022;28(7)doi:10.1038/s41591-022-01833-z
    OpenUrlCrossRef
  55. 55.↵
    D VV, C VU, L B, et al. Adapted large language models can outperform medical experts in clinical text summarization. Nature medicine. 2024 Apr 2024;30(4)doi:10.1038/s41591-024-02855-5
    OpenUrlCrossRef
  56. 56.↵
    C P, J L, R V, V G, E W, M B. Large language models for preventing medication direction errors in online pharmacies. Nature medicine. 04/25/2024 2024;doi:10.1038/s41591-024-02933-8
    OpenUrlCrossRef
  57. 57.↵
    Yang R, Tan, TF, Lu, W, Thirunavukarasu, AJ, Ting, DSW, Liu, N . Large language models in health care: development, applications, and challenges . . Health Care Sci 2023;2:255 – 263.
    OpenUrl
  58. 58.↵
    B N, G Q, BK B-J, et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nature medicine. 2020 Sep 2020;26(9)doi:10.1038/s41591-020-1041-y
    OpenUrlCrossRefPubMed
  59. 59.↵
    GS C, KGM M, P D, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ (Clinical research ed). 04/16/2024 2024;385doi:10.1136/bmj-2023-078378
    OpenUrlCrossRef
  60. 60.↵
    B V, M N, B C, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature medicine. 2022 May 2022;28(5)doi:10.1038/s41591-022-01772-9
    OpenUrlCrossRefPubMed
  61. 61.↵
    Ning Y, Teixayavong S, Shang Y, et al. Generative Artificial Intelligence in Healthcare: Ethical Considerations and Assessment Checklist. 2023/11/02 2023;
  62. 62.↵
    ZSY W, N W, J L, S U. A large dataset of annotated incident reports on medication errors. Scientific data. 02/29/2024 2024;11(1)doi:10.1038/s41597-024-03036-2
    OpenUrlCrossRef
  63. 63.↵
    Rights of the Individual. 2023;
  64. 64.
    Organization TWH. Regulation and Prequalification: What is Pharmacovigilance? Accessed 20 Feb 2024,
Back to top
Posted September 14, 2024.
Download PDF
Data/Code
Email
Generative AI and Large Language Models in Reducing Medication Related Harm and Adverse Drug Events – A Scoping Review
Jasmine Chiat Ling Ong, Chen Michael, Ning Ng, Kabilan Elangovan, Nichole Yue Ting Tan, Liyuan Jin, Qihuang Xie, Daniel Shu Wei Ting, Rosa Rodriguez-Monguio, David W. Bates, Nan Liu
medRxiv 2024.09.13.24313606; doi: https://doi.org/10.1101/2024.09.13.24313606
Twitter logo Facebook logo LinkedIn logo Mendeley logo
Citation Tools

Subject Area