Overlooked bias with thermometer evaluations using quickly retaken temperatures in EHR: applications to axillary, oral, temporal artery, and tympanic thermometry

Abstract

Studies related to COVID-19 increasingly use electronic health records (EHR) to obtain large-scale evidence. However, EHR-based research must be performed with care because it can involve new study design problems that are unfamiliar to much of the medical community. Haimovich et al. (2020) sought to inform COVID-19 practice by evaluating temporal artery thermometers (TATs). They retrospectively searched EHR for temperatures measured twice within 15 minutes, including once with a TAT. The TAT often disagreed with reference measurements, so Haimovich et al. concluded TATs perform poorly. Here, we extended Haimovich et al.’s study design to all other major thermometer types using the eICU Collaborative Research Database. We retrospectively identified 80,065 pairs of quickly retaken temperatures from 24,765 adult U.S. critical care patients treated in 2014-2015. We found that oral, tympanic, and axillary thermometers disagreed with reference measurements as much as TATs did. Moreover, all thermometer types showed unprecedentedly worse agreement than observed in research using other study designs: every thermometer type broke ±0.9°F (±0.5°C) limits of clinically acceptable agreement by >2-fold and no type satisfied basic standards for repeatability. A natural explanation for these findings is that clinicians often retook temperatures within minutes because of user or patient errors during measurement, such as probe misplacement or patient movement. This means that quickly retaken EHR measurements do not reflect device accuracy or precision in correct use and, contrary to Haimovich et al.’s conclusions, should not be used to evaluate thermometer performance or revise COVID-19 fever thresholds. Our study provides an illustrative example of unexpected study design problems that can undermine EHR-based research.