Early Identification of Patients with Acute Gastrointestinal Bleeding using Electronic Health Record Phenotyping

Introduction

Acute gastrointestinal bleeding (GIB) is the most common gastrointestinal diagnosis requiring hospital admission in the United States.¹ Guidelines for upper and lower GIB recommend risk stratification of patients, including the use of risk assessment scores.^2-4 Although many risk stratification tools have been developed and validated, they are not commonly used in real-world clinical practice partly because providers must manually enter a variety of variables into the scoring system. Widespread electronic health record (her) adoption makes it possible to automatically deploy risk stratification scores within the clinical workflow for acute GIB; however, in order to embed risk stratification models into EHRs and deploy them in real time, patients must first be correctly identified. This process of accurately identifying patients, called phenotyping, is used for any study that seeks to reliably group patients with a specific diagnosis or condition, from surveillance studies to comparative effectiveness research. Phenotyping is typically the first step in developing and validating risk stratification models within EHRs.⁵ Such processes have been used to improve the accuracy of case definition of inflammatory bowel disease patients as Crohn’s disease or ulcerative colitis, as well as to facilitate clinical trial recruitment and deploy randomized controlled trials.^{6, 7}

Unlike many conditions that require multiple elements of the record (laboratory testing, reported symptoms, and biometrics such as vital signs) for diagnosis, acute GIB is a condition that can be directly and clearly identified using a limited number of terms by patient report or provider evaluation. To our knowledge, however, no previous study has explored the early identification of patients with acute GIB with an EHR-based model or its implementation within standard EHR workflow.

Phenotypes utilize both structured and unstructured data and are typically used retrospectively after the clinical encounters have ended (e.g. ICD codes are a popular component of phenotypes). However, if the goal is to identify patients that would benefit from predictive models tailored for a particular condition, EHR phenotypes must use data elements generated during the visit. The Systematized Nomenclature of Medicine (SNOMED) is an international comprehensive clinical terminology that is the standard for encoding patient conditions in the EHR. Other approaches for phenotyping can be decision rules using specific data elements (e.g. triage diagnosis) or a machine learning (ML) approach to utilize unstructured clinical text through natural language processing (NLP). ML models use computational modeling to learn from data and their performance can improve with increasing amounts of data. NLP is a tool used to extract data from narrative text, and provides a comparison to decision rules. For prediction, however, a “screening” phenotype using data elements likely to be entered close to real time may be better than a tool that may be delayed (e.g. due to delays in notewriting).⁸

Our study aimed to accurately identify patients with acute GIB reported or witnessed in the emergency department, such that the identification of this phenotype can occur in real time in the EHR to subsequently launch predictive models for risk stratification. We chose to use the current standard for phenotyping patients, SNOMED, as the comparator, even though SNOMED does not provide real-time phenotyping in the emergency department.

Methods

We began with creating a sensitive data mart, a patient dataset selected using specified criteria, to screen for all patients presenting with GIB from 2014 to 2017 in the Yale New Haven Health System electronic health record (Epic). Creation of a sensitive data mart allows exclusion of patients with no evidence of the phenotype and to adequately handle the volume, heterogeneity, and velocity of data.⁹ To create the data mart, we screened for patients with data that suggested the phenotype of overt gastrointestinal bleeding. In order to maximize the capture of relevant data, we defined the process by which patients were evaluated in the emergency department and common time periods for data entry. We also identified points at which a diagnosis would be entered in the electronic health record throughout the hospital stay: hospital problem list, encounter diagnosis, admission diagnosis, and hospital billing diagnosis.

There were 4 categories of screening criteria, which were selected based on existing identifiers in the EHR used to denote gastrointestinal bleeding (Figure 1).

• Download figure
• Open in new tab

Figure 1:

Screening criteria used to create data mart for acute gastrointestinal bleeding

Development of EHR GIB Phenotypes

We developed rule-based algorithms and a machine-learning based algorithm using NLP and compared their performance to the SNOMED-only classification (Figure 2).

• Download figure
• Open in new tab

Figure 2:

Rule-based algorithms and machine learning (Natural Language Processing) algorithm

To determine the specific data elements included in the rule-based algorithm, we analyzed the clinical workflow to identify two points where relevant diagnosis or symptom data was entered into the EHR. The first point of data entry was at triage, where a nurse selected presenting diagnoses from a drop down list of pre-specified diagnoses. We reviewed all triage diagnoses to identify any that were gastrointestinal diagnoses. The second data entry point was the review of systems section in the note template used for all ED patients, which contains elements referring to overt gastrointestinal bleeding. Both the triage diagnosis and review of systems provide structured data fields that can be used to identify patients as either 0 (not present) or 1 (present). We hypothesized that terms specific to gastrointestinal bleeding may have improved performance and therefore created two decision rules. Decision rule 1 was positive if the ROS field (e.g. hematemesis, blood in stool) was positive or if any GI triage term was positive. Decision rule 2 was positive if the ROS field was positive or GIB triage terms were positive.

The NLP approach includes preprocessing of the unstructured text in notes written by physician providers in the Emergency Department using ScispaCy, a Python software library used for advanced NLP that allows for breaking down text into smaller unique parts, by removing adverbs, pronouns, conjunctions, punctuation, participles and spaces.¹⁰ The text was then vectorized into bigrams and classified using a random forest classifier, a strategy that has been applied previously to radiology reports and other biomedical text data.^11,12 We also performed a sensitivity analysis for hematemesis or melena and hematochezia since there is a distinction between upper gastrointestinal bleeding and lower gastrointestinal bleeding.

Results

Discussion

This is the first study to develop an EHR phenotype for acute gastrointestinal bleeding that could be used in the emergency room to deploy risk scores to inform determine level of care and inform clinical management decisions. (Figure 3) We found that the decision rule that combined bleed-specific terms at initial triage by a nurse and bleed-specific terms in the emergency department provider’s review of systems had the highest PPV (85%) on external validation with a sensitivity of 84% to identify patients presenting with acute GIB in the emergency department. This decision rule can be used to trigger alerts that can help triage patients into high and low risk.

• Download figure
• Open in new tab

Figure 3:

Schematic showing how the electronic health record (EHR) phenotyping rule fits into the provider workflow to find patients with gastrointestinal bleeding and deploy risk scores to assist decision making.

We chose the decision rule because in practice we want to deploy a risk prediction tool for patients who actually have acute GIB, which means we want a low false positive rate (high specificity and positive predictive value). Sensitivity should also be as high as possible to ensure that most patients with acute GIB are detected. A positive predictive value of >75% has been reported as “high,” and the PPV for the highest performing decision rule was 91% on internal validation and 85% on external validation.^13-18

Traditionally no method exists to identify patients with gastrointestinal bleeding other than using diagnostic codes (ICD-9, ICD-10) from the billing diagnosis list. By definition, these diagnostic codes are useless for detecting patients presenting with acute GIB at the point of care because they generally are not entered into the EHR until much later in the hospital course, well after initial identification and risk assessment is needed. Additionally, ICD codes may not detect all patients who have bleeding, as it may be reported but not coded in their billing diagnosis. SNOMED may be better than diagnostic codes alone—it is a terminology that includes both billing codes and other related text.^19,20 Our decision rule outperforms the baseline SNOMED methodology even when sampled over multiple areas in the EHR, including the billing diagnosis list. Interestingly, introduction of NLP over the entire text of the emergency department provider notes has a much higher sensitivity but even in conjunction with the decision rules does not have high specificity. Furthermore, the availability of the entire text is variable, since providers may choose to complete documentation up to 24 hours after the actual visit.

The decision for need, timing, and type of endoscopy may be based on presumptive diagnosis (e.g. upper endoscopy versus colonoscopy), but the initial management is similar for both UGIB and LGIB. The level of care, resuscitation, and optimization of other active co-morbidities are relevant prior to any endoscopic evaluation, EGD and colonoscopy are both recommended for most patients admitted with GIB. The sensitivity analysis suggested that neither decision rule performs well to identify symptoms associated with UGIB (hematemesis or melena) or LGIB (hematochezia). The NLP tool had a higher PPV both UGIB and LGIB, but is not suited for prospective deployment in real time to identify patients with GIB. The sensitivity of NLP for upper GIB was incredibly low (27%), reflecting the fact that the tool would incorrectly classify more than half of patients with the symptoms as negative. The NLP tool performed better for lower GIB, and could be explored further in a separate study. The role of novel tools such as NLP continues to advance, and we are hopeful that new techniques and tools will lead to improved performance.