The effect of sepsis recognition on telemedicine use and bundle completion in rural emergency department sepsis treatment ========================================================================================================================= * Anna M. Kaldjian * J. Priyanka Vakkalanka * Uche Okoro * Cole Wymore * Karisa K. Harland * Kalyn Campbell * Morgan B. Swanson * Brian M. Fuller * Brett Faine * Anne Zepeski * Edith A. Parker * Luke Mack * Amanda Bell * Katie DeJong * Kelli Wallace * Keith Mueller * Elizabeth Chrischilles * Christopher R. Carpenter * Michael P. Jones * Marcia M. Ward * Nicholas M. Mohr ## ABSTRACT **Purpose** Provider-to-provider emergency department telehealth (tele-ED) has been proposed to improve rural sepsis care. The objective of this study was to measure the association between sepsis documentation and tele-ED use, treatment guideline adherence, and mortality. **Materials and Methods** This analysis was a multicenter (n=23) cohort study of sepsis patients treated in rural emergency departments (EDs) that participated in a tele-ED network between August 2016 and June 2019. The primary exposure was whether sepsis was recognized in the local ED, and the primary outcome was rural tele-ED use, with secondary outcomes of time to tele-ED use, 3-hour guideline adherence, and in-hospital mortality. **Results** Data from 1,146 rural sepsis patients were included, 315 (27%) had tele-ED used, and 415 (36%) had sepsis recognized in the rural ED. Sepsis recognition was not independently associated with higher rates of tele-ED use (adjusted odds ratio [aOR] 1.23, 95% CI 0.90–1.67). Sepsis recognition was associated with earlier tele-ED activation (adjusted hazard ratio [aHR] 1.69, 95% CI 1.34-2.13), lower 3-hour guideline adherence (aOR 0.73, 95% CI: 0.55-0.97), and lower in-hospital mortality (aOR 0.72, 95% CI: 0.54-0.97). **Conclusions** Sepsis recognition in rural EDs participating in a tele-ED network was not associated with tele-ED use. Keywords * telemedicine * emergency service * hospital * sepsis * hospitals * rural ## Introduction Sepsis contributes to 850,000 emergency department (ED) visits annually in the United States (U.S.).1 Of those with community-onset sepsis, mortality is 13%, making sepsis a leading cause of death in U.S. hospitals and a significant public health problem.2, 3 Sepsis also exhibits a significant volume-outcome relationship, with hospitals that have lower sepsis volumes having higher in-hospital sepsis mortality than those with high volumes.4, 5 Vague and varied symptoms at presentation make sepsis diagnosis challenging. Up to one-third of patients present without classic symptoms, and older and immunocompromised patients are particularly at risk for delayed recognition—possibly affecting early treatment and clinical outcomes.6–8 Variability in sepsis presentations has prompted the use of standardized tools to improve recognition, but universal recognition remains elusive.9 The Surviving Sepsis Campaign (SSC) publishes guidelines that recommend sepsis treatments, and timely high-quality sepsis care has been shown to improve clinical outcomes.10, 11 Telehealth has been proposed as one way to mitigate sepsis disparities in low-volume hospitals, bringing tertiary care expertise into low-volume EDs.12 Telehealth has been used in EDs for acute conditions such as ischemic stroke, myocardial infarction, and trauma, and it may impact care in sepsis.12–18 The objective of this study was to determine whether sepsis recognition in the ED is associated with ED-based telehealth (tele-ED) use, increased SSC bundle adherence, and decreased mortality. We performed a secondary analysis of the TELEmedicine as a Virtual Intervention for Sepsis in Emergency Departments (TELEVISED) study to identify (1) the proportion of rural sepsis patients recognized in local ED documentation, (2) predictors of local sepsis recognition, (3) the association between sepsis recognition, tele-ED use, and tele-ED timing, (4) association between sepsis recognition and SSC bundle adherence, and (5) the association between sepsis recognition and in-hospital mortality. We hypothesized that sepsis recognition would be associated with increased tele-ED use, higher SSC bundle adherence, and lower in-hospital mortality. ## Methods ### Study Design and Setting This analysis is a cohort study conducted as a secondary analysis of the TELEVISED study, which was a retrospective, multicenter (n=23) study designed to measure the association between tele-ED use and clinical outcomes conducted in rural EDs between August 1, 2016 and June 30, 2019. Participating hospitals were rural as defined by the Federal Office of Rural Health Policy and took part in a single provider-to-provider tele-ED network.19 Detailed methods of the study and characteristics of included sites have been reported elsewhere, and the results of the main study have been previously published.20, 21 This study was approved by local institutional review boards under waiver of informed consent, and it is reported in accordance with the Strengthening Observational Studies in Epidemiology (STROBE) statement.22 ### Selection of Patients An electronic health record (EHR) data query was used to identify all adults (age≥18 years) diagnosed with sepsis in participating rural EDs during the study period. All of the following four inclusion criteria were required to meet our operational definition of ED sepsis as we have used previously (to account for explicit sepsis diagnosis codes in the ED being insensitive): (1) inpatient discharge diagnosis of sepsis by *International Classification of Diseases, 10th edition, Clinical Modification* (ICD-10-CM) codes20, (2) infection suspected in the ED based on documentation in the ED clinical note (EHR), (3) organ failure in the ED (defined as a Sequential Organ Failure Assessment Score [SOFA] of 2 or greater or a change of at least 2 in patients with chronic disease), and (4) at least 2 systemic inflammatory response syndrome (SIRS) criteria in the ED.23, 24 Notably, patients could meet this criteria without sepsis being the ED diagnosis—even if formal sepsis diagnostic criteria were met.23, 25 We excluded any patient with no local diagnoses clearly documented in the EHR, as these medical records were likely incomplete. Data were abstracted by trained data abstractors using a structured data collection instrument according to the methods outlined by Kaji, et al.26 ### Interventions All EDs participated in a single hub-and-spoke tele-ED network and used a standard nurse-directed screening protocol. The local screening protocol was positive if (1) the local nurse “suspected infection” and (2) met 2 of the following criteria: temperature <96.8°F or >100.9°F, pulse >90 beats per minute, systolic blood pressure <90 mmHg, mean arterial blood pressure <65 mmHg, oxygen saturation <90%, respiratory rate >20 breaths/minute, lactate >2 mmol/L, white blood cell count <4,000 cells/μL or >12,000 cells/ μL. Local patients were screened by a triage nurse, and positive screens were reported to the local treating clinician. Tele-ED use was encouraged for patients screening positive, and we described the full screening protocol previously.27 The tele-ED network was an on-demand monthly subscription-based provider-to-provider real-time consultation service based in Sioux Falls, SD that could be activated by local providers or nurses 24 hours per day by pressing a button on the wall of the patient treatment room. Tele-ED consultation was provided by a board-certified emergency physician and an experienced ED nurse through a high-definition video connection. These consultants had access to the local EHR and were able to review records, place orders, provide real-time on-camera guidance, and arrange interhospital transfer when necessary. ### Exposures and Outcomes Our primary exposure was whether sepsis was recognized in the rural ED (determined by EHR documentation), and our primary outcome was tele-ED use. Secondary outcomes were timing of tele-ED activation, SSC 3-hour bundle adherence (i.e., measure lactate, draw blood cultures before antibiotics, administer broad-spectrum antibiotics, and administer 30 mL/kg crystalloid fluid bolus if lactate is >4 mmol/L or systolic blood pressure is <90 mmHg), and in-hospital mortality.10 ### Determination of Sepsis Recognition Sepsis recognition was determined locally in the index ED. To determine whether sepsis was recognized locally, we conducted free-text analysis of ED diagnoses by two independent reviewers, who were trained using sample cases. All recorded diagnoses from the ED “impression” (not including differential diagnoses documented in the medical decision-making) were reviewed and classified into one of four mutually exclusive groups: (1) sepsis, (2) infection but no diagnosis of sepsis, (3) signs or symptoms that could be consistent with sepsis but without diagnosis of sepsis or infection, and (4) other unrelated diagnoses only. In this classification, sepsis included sepsis, septic shock, severe sepsis, or any other form of systemic response to infection (including SIRS as a diagnosis), but not including documentation of localized infection alone (e.g., septic arthritis). For example, a patient with an ED impression of “pneumonia” would be classified as “infection;” a patient with an ED impression of “hypoxia, hypotension, and acute kidney injury” would be diagnosed as “signs or symptoms consistent with sepsis;” and a diagnosis of “atrial fibrillation with rapid ventricular response” would be diagnosed as “other unrelated diagnoses only.” Notably, patients were included in the overall study if local clinicians documented in their medical decision-making that they suspected infection (e.g., as a differential diagnosis), but cases were not classified as infection unless an infection was listed in the “impression” portion of the note. Disagreements between the two reviewers were resolved by a third reviewer. ### Definitions Fever was defined as temperature >38.0°C and hypothermia was defined as temperature <36.0°C. Hypotension was defined as systolic blood pressure <90 mmHg. Elevated lactate was defined as lactate >2 mmol/L. Tachycardia was defined as heart rate >100 beats/minute. “Time zero” for time-based analyses was defined as the time of the initial set of vital signs in the ED when patients first met SIRS criteria for bundle adherence metrics and the time of ED arrival for time-to-tele-ED activation. ### Statistical Analysis We calculated the percentage of locally recognized sepsis cases as the percentage of total included patients with an ED diagnosis of sepsis. Then we measured the association between recognized sepsis and variables of interest, using univariate logistic regression. Variables of interest were selected *a priori* and included source of infection, fever, hypothermia, hypotension, tachycardia, elevated lactate, surgery during hospitalization, SOFA score, and ED provider type (advanced practice provider [APP, either solo provider or provider with on-site physician] vs. physician only). We used multivariable generalized estimating equations (GEE) with a logit link to identify predictors of locally recognized sepsis using our *a priori* variables. We then removed variables based on statistical criteria using the Quasi-Likelihood Information Criterion (QIC) to achieve parsimony, we required tele-ED use to remain in the model, and we included hospital identifier as a random effect to account for clustering.28 All covariates were categorical, and we confirmed model goodness of fit.29 We screened for collinearity and interaction terms. Then we constructed another GEE model using similar methods to measure the association between sepsis recognition (exposure) and 3-hour SSC bundle adherence (outcome) and a separate model to measure the association between sepsis recognition (exposure) and in-hospital mortality (outcome). Among patients who had tele-ED used, we compared time-to-tele-ED activation using Kaplan-Meier curves and the log-rank test to identify whether patients with recognized sepsis had faster tele-ED activation. Data for this analysis were right censored if patients left the emergency department or died. Then, we used a Cox proportional hazards model adjusted for the same covariates to measure the association (adjusted hazard ratio [aHR]) between sepsis recognition (exposure) and time-to-tele-ED (outcome). We assessed the proportional hazards assumption of the adjusted model using a global test based on the Schoenfeld residuals. Analyses were conducted as complete case analysis without imputation, but missing values on the variables of interest were rare. All statistical procedures were conducted using two-tailed tests, p<0.05 was considered statistically significant, and analyses were conducted using SAS v 9.4 (SAS Institute, Cary, NC). ## Results ### Baseline Characteristics We included 1146 patients (**Figure 1**), and 315 (27%) had tele-ED used. Patients had a median age of 72 years (interquartile range [IQR] 62 – 82), and 517 (45%) were female. The most common source of infection was pneumonia (n=619, 54%), and the median ED sequential organ failure assessment (SOFA) score was 4 (IQR 3 – 5). A total of 347 (30%) patients were transferred to another hospital, and 111 (10%) patients died in the hospital. Baseline characteristics are detailed in **Table 1**. ![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/13/2023.08.09.23293892/F1.medium.gif) [Figure 1.](http://medrxiv.org/content/early/2023/08/13/2023.08.09.23293892/F1) Figure 1. Selection of participants. View this table: [Table 1.](http://medrxiv.org/content/early/2023/08/13/2023.08.09.23293892/T1) Table 1. Characteristics of participants. IQR, interquartile range; kg, kilograms; m, meters, 95% CI, 95% confidence interval; SOFA, Sequential Organ Failure Assessment; APP, advanced practice provider; COPD, chronic obstructive pulmonary disease ### Locally recognized sepsis In our sample, 415 (36%) had sepsis recognition documented in the local ED. Other non-sepsis local diagnoses included infection (n=624, 54%); signs, symptoms, or findings consistent with sepsis (n=85, 7%); and an unrelated diagnosis only (n=22, 2%). Sepsis was more likely to be diagnosed in patients who presented with fever (41% vs. 33%, difference 8%, 95% CI 2–14%), hypotension (63% vs. 34%, difference 29%, 95% CI 19–39%), or higher SOFA scores (53% with SOFA 7 or greater vs. 34% with SOFA 2–6, difference 19%, 95% CI 9–29%). Sepsis recognition was higher in those treated by an APP (with or without onsite physician supervision) compared to those treated by a physician alone (44% vs. 35%, difference 9%, 95% CI 1–17%). Tele-ED was used more frequently in those with sepsis recognition compared with those without (45% vs. 23%, difference 13%, 95% CI 7–19%). In multivariable modeling, fever, hypotension, tachycardia, elevated lactate, and APP-directed care (both alone and with an onsite physician) remained predictive of increased odds of locally recognized sepsis, and pneumonia (compared with all other infection sources) was associated with decreased local recognition of sepsis. Tele-ED use was not associated with sepsis recognition in multivariable modeling (aOR 1.23, 95% CI 0.90–1.67, **Figure 2**). ![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/13/2023.08.09.23293892/F2.medium.gif) [Figure 2.](http://medrxiv.org/content/early/2023/08/13/2023.08.09.23293892/F2) Figure 2. Factors predicting local recognition of sepsis. This forest plot depicts the association between signs and symptoms and local recognition of sepsis. Grey circles represent unadjusted values and black squares represent adjusted values. Variables used in statistical adjustment include all those variables shown in this forest plot. *ED, emergency department* The hazard function associated with tele-ED activation was 64% earlier in those with recognized sepsis versus those without (unadjusted HR 1.64, 95% CI 1.31–2.06), and this difference persisted when adjusted for ED disposition and provider type (aHR 1.69, 95% CI 1.34–2.13, **Figure 3**). ![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/13/2023.08.09.23293892/F3.medium.gif) [Figure 3.](http://medrxiv.org/content/early/2023/08/13/2023.08.09.23293892/F3) Figure 3. Time to tele-ED use, stratified by sepsis recognition. This figure depicts the Kaplan-Meier survival plot of time to tele-ED consultation. Data are right censored if patients left the ED or died. *ED, emergency department; APP, advanced practice provider*. ### SSC bundle adherence and in-hospital mortality SSC 3-hr bundle adherence was similar between those with and without locally recognized sepsis in bivariate analyses (15% vs. 12%, difference 2.9%, 95%CI −1.3–7.1%). After covariate adjustment, sepsis recognition was associated with lower 3-hr bundle adherence (aOR 0.73, 95% CI 0.55–0.97). In-hospital mortality was similar in patients who had local sepsis recognition (11% vs. 9%, difference 2.5%, 95%CI −1.2–6.3%); however, after adjusting for covariates, in-hospital mortality was significantly lower in those with local sepsis recognition (aOR 0.72, 95% CI 0.54–0.97, **Figure 4**). ![Figure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2023/08/13/2023.08.09.23293892/F4.medium.gif) [Figure 4.](http://medrxiv.org/content/early/2023/08/13/2023.08.09.23293892/F4) Figure 4. Surviving Sepsis Campaign (SSC) 3-hour bundle adherence and in-hospital mortality, stratified on sepsis recognition. This figure depicts the association between local sepsis recognition, SSC bundle adherence (3-hour), and in-hospital mortality. On the left panel, the raw adherence numbers are reported (gray bars represent recognized sepsis, and black bars represent unrecognized sepsis). On the right panel, the unadjusted and adjusted odds ratios of local sepsis recognition and each outcome are presented (log scale, white circles represent unadjusted analyses, and black squares represent adjusted analyses). Error bars represent 95% confidence intervals. Note that the point estimate and confidence interval for the unadjusted estimate for the 3-hour bundle is hidden by the symbol, but it is OR 0.97 (95% CI 0.95, 1.00). ## Discussion In our analysis of a multicenter cohort study in rural EDs, we found that tele-ED use was more common in those recognized as having sepsis. However, adjusting for factors associated with sepsis recognition, tele-ED use was not independently associated with sepsis recognition. Furthermore, we observed that a minority of sepsis patients in participating rural hospitals were recognized as having sepsis—regardless of tele-ED use. In our prior report of the TELEVISED study, we showed that tele-ED use was not associated with improved clinical outcomes or SSC bundle adherence.21 The finding from this analysis is relevant, because under-recognition could be a key explanation for why our prior work did not show the differences in processes and outcomes we hypothesized we might observe. Sepsis diagnosis—especially for patients with low illness severity—can be difficult, and under-recognition is an obstacle to appropriate and timely sepsis care.7, 30 In a cohort study of 654 ED sepsis patients, 37% presented with vague symptoms, and the absence of explicit, traditional symptoms of sepsis was associated with delays in antibiotic therapy and higher mortality.6 Others have associated misdiagnosis with delayed treatment and worse clinical outcomes.31 This finding parallels our results in that sepsis recognition was associated with decreased adjusted in-hospital mortality, but we additionally found that the failure to recognize sepsis was associated with a lower frequency of tele-ED use.18 Many patients in whom sepsis was not recognized were appropriately diagnosed as having pneumonia, respiratory failure, or kidney injury—all diagnoses that these patients clearly had. One of the reasons sepsis diagnoses may be missed is that sepsis may be a secondary syndrome diagnosed in someone who may appropriately have a primary diagnosis of infection.32 Biomarkers have been proposed as one strategy to improve recognition of the constellation of findings that suggest a host immune response and may define sepsis, and EHR-based algorithms and ICU-based telehealth screening have also been proposed.33–35 Provider-to-provider telehealth has been viewed as a major advantage for managing rare, life-threatening conditions in low-volume centers.36 However, our findings highlight one significant shortcoming of this approach—when tele-ED use is low, appropriate on-demand tele-ED activation requires accurate local diagnosis.37 Under-recognition of sepsis has been commonly reported.31, 38, 39 Similar patterns have emerged in stroke, myocardial infarction, and aortic dissection, where treatment paradigms are clear and under-recognition is associated with morbidity and mortality.40–42 Like other time-sensitive acute care conditions, sepsis treatment has been operationally protocolized, so tele-ED’s greatest potential to improve guideline adherence and outcomes may be from its ability to identify sepsis and initiate a predefined treatment bundle. The fact that remote diagnostic surveillance screening is not a feature of most current tele-ED programs is a significant concern, and it may be a call to action to implement EHR-based and telemetry-based strategies to incorporate into tele-ED networks screening technology for conditions not recognized by local providers. Informatics and machine learning-based tools continue to improve, and providing health system-level surveillance—even remotely—may be one way to amplify the effect of tele-consultation by remote experts.43 Our study has several limitations. First, we relied on EHR documentation to indicate recognition of sepsis or infection, and such documentation may miss cases accurately recognized (but not adequately documented) by the treating clinician. Lack of documentation in the medical record has been hypothesized to decrease case capture for the sepsis core measure, but as most of our sites do not voluntarily report sepsis quality measures44, we think this factor is not likely relevant in our sample. Second, our observational study cannot be used to imply causality, because we do not know whether tele-ED consultation is caused by sepsis recognition, diagnosis is caused by tele-ED consultation, or if clinical features that drive tele-ED consultation are coincidentally associated with accurate sepsis diagnosis. Finally, our multicenter study was conducted in a single tele-ED network that used common procedures at all sites, which may affect the generalizability of our results. In conclusion, sepsis recognition in rural EDs participating in a tele-ED network was not independently associated with tele-ED use, but recognition was associated with decreased adjusted in-hospital mortality. Future work will seek to identify ways to improve rural sepsis recognition using automated methods and tele-ED surveillance, and to identify barriers to effective tele-ED use. ## Data Availability All data produced in the present study are available upon reasonable request to the authors. ## ACKNOWLEDGEMENTS The authors would like to acknowledge Sydney Krispin, MA, MPH for her editorial assistance in preparing this manuscript. ## Footnotes * **Institution Where the Work was Performed:** University of Iowa Carver College of Medicine, Iowa City, Iowa. * **Competing Interests and Source of Funding:** LM, AB, and KD are employed by an organization that provides direct telehealth services. All other authors report no competing interests. This study was funded by the Agency for Healthcare Research and Quality (K08HS025753), the Institute for Clinical and Translational Science at the University of Iowa through a grant from the National Center for Advancing Translational Sciences at the National Institutes of Health (UL1TR002537), and the University of Iowa Department of Emergency Medicine. Dr. Mohr is additionally supported by funding from the Rural Telehealth Research Center with funding from the Health Resources and Services Administration (U3GRH40003). These contents are solely the responsibility of the authors and do not necessarily reflect the views of the Agency for Healthcare Research and Quality, the Health Resources and Services Administration, or any other funding agency. Funding agencies had no role in the study design, data analysis or interpretation, writing the report, or the decision to publish. * Received August 9, 2023. * Revision received August 9, 2023. * Accepted August 13, 2023. * © 2023, Posted by Cold Spring Harbor Laboratory The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission. ## References 1. 1.Wang HE, Jones AR, Donnelly JP. Revised National Estimates of Emergency Department Visits for Sepsis in the United States. Crit Care Med. 2017;45:1443–1449. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 2. 2.Rhee C, Dantes R, Epstein L, et al. Incidence and Trends of Sepsis in US Hospitals Using Clinical vs Claims Data, 2009-2014. JAMA. 2017;318:1241–1249. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2017.13836&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 3. 3.Minhas AMK, Salah HM, Khan MS, et al. Most Common Causes of Hospitalization Associated with Inpatient Mortality in the United States Between 2005-2018. The American Journal of the Medical Sciences. 2022;363:459–461. 4. 4.Gaieski DF, Edwards JM, Kallan MJ, Mikkelsen ME, Goyal M, Carr BG. The relationship between hospital volume and mortality in severe sepsis. Am J Respir Crit Care Med. 2014;190:665–674. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1164/rccm.201402-0289OC&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25117723&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 5. 5.Kocher KE, Haggins AN, Sabbatini AK, Sauser K, Sharp AL. Emergency department hospitalization volume and mortality in the United States. Ann Emerg Med. 2014;64:446–457.e446. 6. 6.Filbin MR, Lynch J, Gillingham TD, et al. Presenting Symptoms Independently Predict Mortality in Septic Shock: Importance of a Previously Unmeasured Confounder. Crit Care Med. 2018;46:1592–1599. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/CCM.0000000000003260&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=29965833&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 7. 7.Gauer R, Forbes D, Boyer N. Sepsis: Diagnosis and Management. Am Fam Physician. 2020;101:409–418. 8. 8.Goodacre S, Fuller G, Conroy S, Hendrikse C. Diagnosis and management of sepsis in the older adult. BMJ. 2023;382:e075585. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjE5OiIzODIvanVsMTRfMS9lMDc1NTg1IjtzOjQ6ImF0b20iO3M6NTA6Ii9tZWRyeGl2L2Vhcmx5LzIwMjMvMDgvMTMvMjAyMy4wOC4wOS4yMzI5Mzg5Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 9. 9.Font MD, Thyagarajan B, Khanna AK. Sepsis and Septic Shock - Basics of diagnosis, pathophysiology and clinical decision making. Med Clin North Am. 2020;104:573–585. 10. 10.Evans L, Rhodes A, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2021. Critical Care Medicine. 2021;49:e1063–e1143. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/ccm.0000000000005337&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 11. 11.Rivers E, Nguyen B, Havstad S, et al. Early Goal-Directed Therapy in the Treatment of Severe Sepsis and Septic Shock. New England Journal of Medicine. 2001;345:1368–1377. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJMoa010307&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11794169&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000172014200002&link_type=ISI) 12. 12.Mohr NM, Campbell KD, Swanson MB, Ullrich F, Merchant KA, Ward MM. Provider-to-provider telemedicine improves adherence to sepsis bundle care in community emergency departments. J Telemed Telecare. 2021;27:518–526. 13. 13.Miller AC, Ward MM, Ullrich F, Merchant KAS, Swanson MB, Mohr NM. Emergency Department Telemedicine Consults are Associated with Faster Time-to-Electrocardiogram and Time-to-Fibrinolysis for Myocardial Infarction Patients. Telemed J E Health. 2020;26:1440–1448. 14. 14.Mohr NM, Harland KK, Chrischilles EA, Bell A, Shane DM, Ward MM. Emergency Department Telemedicine Is Used for More Severely Injured Rural Trauma Patients, but Does Not Decrease Transfer: A Cohort Study. Acad Emerg Med. 2017;24:177–185. 15. 15.Mohr NM, Vakkalanka JP, Harland KK, et al. Telemedicine Use Decreases Rural Emergency Department Length of Stay for Transferred North Dakota Trauma Patients. Telemed J E Health. 2018;24:194–202. 16. 16.Mohr NM, Young T, Harland KK, et al. Telemedicine Is Associated with Faster Diagnostic Imaging in Stroke Patients: A Cohort Study. Telemed J E Health. 2019;25:93–100. 17. 17.Swanson MB, Miller AC, Ward MM, Ullrich F, Merchant KA, Mohr NM. Emergency department telemedicine consults decrease time to interpret computed tomography of the head in a multi-network cohort. J Telemed Telecare. 2021;27:343–352. 18. 18.Tu KJ, Wymore C, Tchangalova N, Fuller BM, Mohr NM. The impact of telehealth in sepsis care: A systematic review. J Telemed Telecare. 2023:1357633x231170038. 19. 19.Defining rural population. Bethesda, Maryland: Health Resources and Services Administration; 2022. 20. 20.Mohr NM, Harland KK, Okoro UE, et al. TELEmedicine as an intervention for sepsis in emergency departments: a multicenter, comparative effectiveness study (TELEvISED Study). J Comp Eff Res. 2021;10:77–91. 21. 21.Mohr NM, Okoro U, Harland KK, et al. Outcomes Associated With Rural Emergency Department Provider-to-Provider Telehealth for Sepsis Care: A Multicenter Cohort Study. Ann Emerg Med. 2023;81:1–13. 22. 22.von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335:806–808. [FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMzUvNzYyNC84MDYiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyMy8wOC8xMy8yMDIzLjA4LjA5LjIzMjkzODkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 23. 23.Bone RC, Balk RA, Cerra FB, et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest. 1992;101:1644–1655. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1378/chest.101.6.1644&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=1303622&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1992HX70800034&link_type=ISI) 24. 24.Vincent JL, Moreno R, Takala J, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22:707–710. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/BF01709751&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=8844239&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=A1996UZ45200017&link_type=ISI) 25. 25.Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). Jama. 2016;315:801–810. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2016.0287&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26903338&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 26. 26.Kaji AH, Schriger D, Green S. Looking through the retrospectoscope: reducing bias in emergency medicine chart review studies. Ann Emerg Med. 2014;64:292–298. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.annemergmed.2014.03.025&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24746846&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 27. 27.Mohr NM, Skow B, Wittrock A, et al. Improving Access to High Quality Sepsis Care in a South Dakota Emergency Telemedicine Network. RTRC Research & Policy Brief. 2017;2017. 28. 28.Wang Y, Murphy O, Turgeon M, et al. The perils of quasi-likelihood information criteria. Stat. 2015;4:246–254. 29. 29.Evans S, Li L. A comparison of goodness of fit tests for the logistic GEE model. Stat Med. 2005;24:1245–1261. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1002/sim.2023&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=15580592&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000228271900008&link_type=ISI) 30. 30.. Seymour CW, Kennedy JN, Wang S, et al. Derivation, Validation, and Potential Treatment Implications of Novel Clinical Phenotypes for Sepsis. Jama. 2019;321:2003–2017. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jama.2019.5791&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 31. 31.Abe T, Tokuda Y, Shiraishi A, et al. In-hospital mortality associated with the misdiagnosis or unidentified site of infection at admission. Critical Care. 2019;23:202. 32. 32.Wiedemann LA. Coding sepsis and SIRS. Journal of AHIMA. 2007;78:76–78. 33. 33.Vincent JL. The Clinical Challenge of Sepsis Identification and Monitoring. PLoS Med. 2016;13:e1002022. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 34. 34.Manaktala S, Claypool SR. Evaluating the impact of a computerized surveillance algorithm and decision support system on sepsis mortality. J Am Med Inform Assoc. 2017;24:88–95. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jamia/ocw056&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27225197&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 35. 35.Rincon TA, Bourke G, Seiver A. Standardizing sepsis screening and management via a tele-ICU program improves patient care. Telemed J E Health. 2011;17:560–564. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1089/tmj.2010.0225&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21718115&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 36. 36. Sharifi Kia A, Rafizadeh M, Shahmoradi L. Telemedicine in the emergency department: an overview of systematic reviews. Z Gesundh Wiss. 2022:1–15. 37. 37.Heppner S, Mohr NM, Carter KD, Ullrich F, Merchant KAS, Ward MM. HRSA’s evidence-based tele-emergency network grant program: Multi-site prospective cohort analysis across six rural emergency department telemedicine networks. PLoS One. 2021;16:e0243211. 38. 38.Cifra CL, Westlund E, Ten Eyck P, Ward MM, Mohr NM, Katz DA. An estimate of missed pediatric sepsis in the emergency department. Diagnosis (Berl). 2021;8:193–198. 39. 39.Newman-Toker DE, Peterson SM, Badihian S, et al. Diagnostic Errors in the Emergency Department: A Systematic Review. Vol 22. Rockville, Maryland: Agency for Healthcare Research and Quality; 2022. 40. 40.Newman-Toker DE, Moy E, Valente E, Coffey R, Hines AL. Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample. Diagnosis (Berl). 2014;1:155–166. 41. 41.Pope JH, Aufderheide TP, Ruthazer R, et al. Missed diagnoses of acute cardiac ischemia in the emergency department. N Engl J Med. 2000;342:1163–1170. [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1056/NEJM200004203421603&link_type=DOI) [PubMed](http://medrxiv.org/lookup/external-ref?access_num=10770981&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000086523700003&link_type=ISI) 42. 42.Lovatt S, Wong CW, Schwarz K, et al. Misdiagnosis of aortic dissection: A systematic review of the literature. Am J Emerg Med. 2022;53:16–22. 43. 43.Schinkel M, Paranjape K, Panday RSN, Skyttberg N, Nanayakkara PWB. Clinical applications of artificial intelligence in sepsis: A narrative review. Computers in biology and medicine. 2019;115:103488. [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2023%2F08%2F13%2F2023.08.09.23293892.atom) 44. 44.Bolte TB, Swanson MB, Kaldjian AM, Mohr NM, McDanel J, Ahmed A. Hospitals That Report Severe Sepsis and Septic Shock Bundle Compliance Have More Structured Sepsis Performance Improvement. J Patient Saf. 2022;18:e1231–e1236.