Use of an EHR to inform an administrative data algorithm to categorize inpatient COVID-19 severity

2.1. Data Sources

Our study used two US RWD sources (see Appendix C for additional detail). First, HealthVerity data comprises medical and pharmacy open claims (sourced in near-real-time from practice management systems, billing systems and claims clearinghouses) and closed claims (sourced from insurance providers and payers), laboratory test history and results, and chargemaster administrative hospital billing data for inpatient and outpatient encounters from all US states. The data include all major payer types (commercial, Medicaid and Medicare). Second, Optum de-identified COVID-19 EHR data includes patients with suspected COVID-19, sourced from medical and administrative encounters from hospitals, emergency departments, outpatient centers, and laboratories from across the US that include diagnosis data, laboratory data with results, procedures, vital sign measurements, prescriptions written, and medications administered. There has been scientific publication of COVID-19 research using both the HealthVerity {Burn 2020; Gordon 2020; Harvey 2021; Murk 2020} and Optum {Hughes 2020} datasets.

In the medical claims of both datasets, procedures are captured via current procedural terminology (CPT), healthcare common procedure coding system (HCPCS), and international classification of disease 10th revision procedural classification system (ICD-10-PCS) codes, diagnoses are captured via ICD-10, clinical modification (ICD-10-CM) codes, and O2 or related supplies indicating an O2 procedure may additionally be captured via revenue codes. In the chargemaster data of HealthVerity, vendor charge codes also capture procedures and diagnoses; although procedures are captured daily while hospitalized, diagnoses are primarily captured at discharge, with some further capture of diagnoses present at admission. In contrast, the Optum de-identified COVID-19 EHR has day-level diagnosis encounters while hospitalized. Comparing the available data in these two data sources offers the opportunity to create a more widely-applicable COVID-19 severity score that can be used in EHR, claims and/or hospital billing data.

2.2. COVID-19 severity levels

The WHO Clinical Progression Scale scores range of COVID-19 severity outcomes from uninfected (score of 0) to dead (10; see Table 1) {WHO Working Group 2020}. We developed a modified version, referred to here as the mWHO score, that restricts to severity levels applicable to hospitalized COVID-19 patients (WHO original scores of 4-9), collapsed into three mutually exclusive categories for neither O2/NIV nor IMV, O2 or NIV without IMV, and any IMV with or without additional support (NEITHER, O2/NIV, and IMV, respectively; see Appendix A for additional mWHO algorithm detail). These categories correspond to the FDA Sentinel’s Moderate, Severe, and Critical categories, respectively, leaving out the Asymptomatic and Mild categories that apply only to non-hospitalized patients {Yih 2020}.

To operationalize the score, patients were assigned to mWHO categories based on the highest severity recorded on the treatment index date, considering both the first day of treatment and the day prior to minimize potential misclassification from situations in which patients received O2 or IMV in other medical settings (e.g., emergency room or ambulance), or cases where the billing date for a procedure was captured at a later calendar date than the procedure was performed due to a lag in reporting.

In each data source, two different algorithms were developed to identify the COVID-19 severity levels (see summary in Table 2). First, we developed procedure-based algorithms that were similar for both data sources (initial-EHR and initial-administrative), except procedure-based charge codes were only applicable for the administrative chargemaster data. Next, to increase the specificity of O2 and IMV identification, we expanded the algorithms to include diagnoses indicating a clinical need for O2/NIV (hypoxia or hypoxemia) or IMV (acute respiratory distress syndrome), among other additions. The expanded-EHR algorithm included available clinical patient vitals and day-level diagnosis data. We then applied the EHR-based learnings to the expanded-administrative algorithm, adapting as needed to analogous data available in the administrative dataset. For example, day-level diagnosis data was not available in the administrative data during the hospitalizations, but we were able to utilize the flag indicating ‘present at admission’ to differentiate the admitting diagnoses from all diagnoses reported at discharge.

2.3. Study population

In each dataset, we identified patients hospitalized with confirmed COVID-19 (diagnosis or positive SARS-CoV-2 laboratory results) between March and August 2020 with at least 1 encounter during the 183-day baseline and no prior CSI use during the 90-day washout period. From these populations, we selected primary cohorts of DEX initiators (DEX+) and a random selection of patients who had not or had not yet initiated any CSI (CSI-) matched 1:1 on date of DEX initiation, age, sex, Charlson-Quan comorbidity score over the 183-day baseline period {Quan 2005}, days since admission, and the mWHO severity level (NEITHER, O2/NIV, IMV) on treatment index (see NCT04926571 protocol for additional detail). The primary analysis cohorts matched on mWHO severity using the initial algorithms, while secondary analysis cohorts were rematched using the expanded algorithms.

2.4. Approach / statistical analysis

The primary analysis evaluated the treatment-specific distributions of patients in each mWHO severity level, according to both the initial algorithms that patients were matched on and the expanded algorithms. This was done to describe how many of the patients from each initial level would have shifted to a higher severity level via the expanded algorithms (the percent of NEITHER who shifted to O2/NIV or IMV, and the percent of O2/NIV that shifted to IMV). We started with the EHR data and then repeated the same process using the administrative data.

We performed two secondary analyses and one sensitivity analysis largely focusing on the administrative data. The first secondary analysis rematched cohorts in both datasets using the expanded algorithms to illustrate the distribution of patients who qualified for O2/NIV and IMV via each measure component (i.e., procedures, medical diagnosis claims, chargemaster diagnosis data, and revenue codes) using Venn diagrams to understand the relative contribution of each component type. The second compared the distribution of select patient characteristics in each treatment arm of the administrative data cohort in the categories they were matched on using the initial-administrative algorithm to the same cohort recategorized using the expanded-administrative algorithm. Lastly, a sensitivity analysis evaluated the shifts in severity and the relative contribution of each component type among a subset of patients in the administrative data cohort who were early initiators (initiated treatment on the same day or 1 day after the admission date).

All analyses were conducted using the Aetion Evidence Platform® (2021), a software for real-world data analysis, validated for a range of studies. {Wang 2016} Visualizations were created using the tidyverse {Wickham 2020} and plotly {Sievert 2019} packages in R (v4.0.3).

3.1. Primary analyses

In the EHR-based cohort with 1,768 DEX+ and CSI− patient pairs matched using the initial-HER algorithm, 17% were categorized as O2/NIV, 2% as IMV, and 81% as NEITHER (Figure 1). Among those categorized as NEITHER, use of the expanded-EHR algorithm shifted 56% DEX+ and 32% CSI- to O2/NIV, and 5% DEX+ and 3% CSI- to IMV. Among the patients initially categorized as O2/NIV, 10% DEX+ and 8% CSI-patients shifted to IMV.

• Download figure
• Open in new tab

Figure 1. Shifts in COVID-19 severity score in each treatment arm in the EHR-based cohort matched using the initial-EHR algorithm (n=3,536; 1,768 pairs)

In the administrative data cohort with 5,524 DEX+ and CSI− patient pairs matched using the initial-administrative algorithm, 30% were categorized as O2/NIV, 5% as IMV, and 65% as NEITHER (Figure 2). Similar to the EHR-based cohort, among those categorized as NEITHER, use of the expanded-administrative algorithm shifted 54% DEX+ and 28% CSI- to O2/NIV, and 2% DEX+ and 1% CSI- to IMV. Among the patients initially categorized as O2/NIV, 7% DEX+ and 3% CSI− patients shifted to IMV.

• Download figure
• Open in new tab

Figure 2. Shifts in COVID-19 severity score in each treatment arm in the administrative data cohort matched using the initial-administrative algorithm (n=11,048; 5,524 pairs)

3.2. Secondary and sensitivity analyses

After matching using the expanded-administrative algorithm, the sample size slightly decreased to 5,337 DEX+ and CSI− patient pairs with 61% categorized as O2/NIV, 7% as IMV, and 32% as NEITHER (Appendix Figure B.1; Appendix Table B.1). Illustrated areas of overlap indicate the distribution of patients with more than one measure component that qualifies them for either O2/NIV or IMV, while areas of non-overlap indicate only one qualifying component. Among the patients categorized as O2/NIV, 49% of both DEX+ and CSI− qualified only due to the added clinical diagnosis of hypoxia or hypoxemia of the expanded-administrative algorithm (i.e., did not have a procedure-based component). In contrast, 27% of DEX+ and 24% of CSI− qualified as IMV due to only the clinical diagnosis of acute respiratory distress syndrome. The inclusion of revenue codes only added 1 DEX+ patient to the O2/NIV group.

Additional analyses of the EHR-based cohort matched using the expanded-EHR algorithm are shown in Appendix B. The addition of clinical diagnoses in this expanded algorithm had a similar impact on the mWHO severity categorization to that seen in the expanded-administrative algorithm. Additionally, patient vitals were responsible for 14% of DEX+ and 27% of CSI− patients categorized as O2/NIV (Appendix Figure B.2).

Compared to the mWHO severity categorization using the initial-administrative algorithm that the patients were matched on, recategorization using the expanded-administrative algorithm showed that there was a significant decrease in the IMV group for heart disease for DEX+ patients (ASD: 0.12; Table 3). There was a treatment-differential change that was greater for DEX+ versus CSI− patients in the NEITHER, O2/NIV, and IMV groups for diabetes (ASD: 0.04 vs 0.02, 0.05 vs 0.02, and 0.08 vs 0.06), NEITHER and O2/NIV groups for age (ASD: 0.08 vs 0.07 and 0.04 vs 0.02), and in the IMV group for heart disease (ASD: 0.12 vs 0.01) chronic pulmonary disease (ASD: 0.09 vs 0.04, and end-stage renal disease (ASD: 0.09 vs 0.07). There was also a treatment-differential change that was less for DEX+ versus CSI− patients in the IMV group for age (ASD: 0.04 vs 0.05).

When the administrative cohort was restricted to early initiators in the sensitivity analysis, similar shifts in mWHO severity and relative contributions of each component type were identified (Figures B.3-B.4).