Abstract
Objectives To determine whether data captured in electronic medical records (eMR) is sufficient to serve as a clinical data source to make a reliable determination of ST elevation myocardial infarction (STEMI) and non-ST elevation myocardial infarction (NSTEMI) and to use these eMR derived diagnoses to validate ICD-10 codes for STEMI and NSTEMI.
Design Retrospective validation by blind chart review of a purposive sample of patients with a troponin test result, ECG record, and medical note available in the eMR.
Setting Two local health districts containing two tertiary hospitals and six referral hospitals in New South Wales, Australia.
Participants N = 897 adult patients who had a hs-troponin test result indicating suspected AMI.
Primary outcome measures Inter-rater reliability of clinical diagnosis (κ) for ST-elevated myocardial infarction (STEMI) and Non-ST elevated myocardial infarction (NSTEMI); and sensitivity, specificity, and positive predictive value (PPV) of ICD-10 codes for STEMI and NSTEMI.
Results The diagnostic agreement between clinical experts was high for STEMI (κ = 0.786) but lower for NSTEMI (κ = 0.548). ICD-10 STEMI codes had moderate sensitivity (Se = 88±6.7), very high specificity (Sp = 99±0.7) and high positive predictive value (PPV = 91±6). NSTEMI ICD-10 codes were lower in each case (Se = 69±6.4, Sp = 96.0±1.5, PPV = 84±6).
Conclusions The eMR held sufficient clinical data to reliably diagnose STEMI, producing high inter-rater agreement among our expert reviewers as well as allowing reasonably precise estimates of the accuracy of administrative ICD-10 codes. However the clinical detail held in the eMR was less sufficient to diagnose NSTEMI, indicated by a lower inter-rater agreement. Efforts should be directed towards operationalising the clinical definition of NSTEMI and improving clinical record keeping to enable an accurate description of the clinical phenotype in the eMR, and thus improve reliability of the diagnosis of NSTEMI using these data sources.
Strengths and limitations of this study
Expert chart review provided a robust evaluation of the reliability and sufficiency of data directly extracted from the EMR for the diagnosis of AMI
Computational interrogation and extraction of the eMR (via SPEED-EXTRACT) allowed us to use a wide selection for inclusion in the sample on the basis of clinical data independent of ICD-10 code, enabling the capture of missed cases (i.e., uncoded AMI) and so determine estimates for the false negative rate and sensitivity
Results were necessarily based on the subset of patients with sufficient clinical data in the eMR. Inferences from this subset to the wider patient pool will be biased when the availability of records varies with diagnosis
At least two sources of uncertainty in the gold reference standard we used are indistinguishable: uncertainty due to poor clinical detail in the eMR, and uncertainty due to a weak operational definition of the diagnosis (e.g., NSTEMI).
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
Funding for the SPEED-EXTRACT study was provided by grants from the Agency for Clinical Innovation, NSW Ministry of Health and Sydney Health Partners. Author CST was supported by a National Health and Medical Research Centre Early Career Fellowship from Australia (#1037275). Author AS was supported by Sydney Health Partners (Harnessing the eMR to improve care of patients with acute chest pain within Sydney Health)
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Northern Sydney Local Health District (NLSHD) Human Research Ethics Committee (HREC), reference: HREC/17/HAWKE/192
All necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Footnotes
↵* Co-first author
Data Availability
Data is available from https://github.com/datarichard/SPEED-EXTRACT-validation
