Abstract
Background Novel applications of telemedicine can improve care quality and patient outcomes. Telemedicine for intraoperative decision support has not been rigorously studied.
Methods This single centre randomised clinical trial (clinicaltrials.gov NCT03923699) of unselected adult surgical patients was conducted between July 1, 2019 and January 31, 2023. Patients received usual care or decision support from a telemedicine service, the Anesthesiology Control Tower (ACT). The ACT provided real-time recommendations to intraoperative anaesthesia clinicians based on case reviews, machine-learning forecasting, and physiologic alerts. ORs were randomised 1:1. Co-primary outcomes of 30-day all-cause mortality, respiratory failure, acute kidney injury (AKI), and delirium were analysed as intention-to-treat.
Results The trial completed planned enrolment with 71927 surgeries (35956 ACT; 35971 usual care). After multiple testing correction, there was no significant effect of the ACT vs. usual care on 30-day mortality [641/35956 (1.8%) vs 638/35971 (1.8%), risk difference 0.0% (95% CI −0.2% to 0.3%), p=0.96], respiratory failure [1089/34613 (3.1%) vs 1112/34619 (3.2%), risk difference −0.1% (95% CI −0.4% to 0.3%), p=0.96], AKI [2357/33897 (7%) vs 2391/33795 (7.1%), risk difference −0.1% (−0.6% to 0.4%), p=0.96], or delirium [1283/3928 (32.7%) vs 1279/3989 (32.1%), risk difference 0.6% (−2.0% to 3.2%), p=0.96]. There were no significant differences in secondary outcomes or in sensitivity analyses.
Conclusions In this large RCT of a novel application of telemedicine-based remote monitoring and decision support using real-time alerts and case reviews, we found no significant differences in postoperative outcomes. Large-scale intraoperative telemedicine is feasible, and we suggest future avenues where it may be impactful.
Introduction
Tele-critical care (TCC), the use of remote monitoring and telemedicine to augment intensive care unit (ICU) services, 1 operates in most academic hospitals2 and in nearly 20% of hospitals with ICU beds.3 Although there are multiple TCC models, 4 contemporary TCC supports bedside teams through a combination of expert consultations, real-time review of emerging problems, continuous remote monitoring, encouragement of best practices, and facilitating communication. 2 TCC support improves care quality and patient outcomes by reducing and overcoming distractions to bedside clinicians. 2,5 Although effect size estimates from quasi-experimental TCC studies have been heterogeneous,6 TCC has repeatedly been associated with improved patient outcomes. 1,2 However, no RCT data has shown a positive impact on patient outcomes.
Although seemingly a natural extension of the adoption of TCC, there are limited applications of or research evaluating remote monitoring models for intraoperative care. Existing case reports focus on intraoperative one-on-one video calls to support low-resource or remote locations. 7–12 This is surprising because critical care and anaesthesia care are closely related fields, with similar physiologic changes and monitoring technology. Like critical care settings, anaesthesia clinicians experience high cognitive load and distractions from complex, high-frequency data combined with procedures and documentation. Decision support from telemedicine may therefore have a similar beneficial role. 13
Therefore, we hypothesized that remote monitoring and support via a telemedicine centre for anaesthesia clinicians similar to current tele-critical care delivery models would improve surgical patient outcomes. Towards this end, we developed the “Anesthesiology Control Tower” (ACT), 14–16 a telemedicine support service for anaesthesia clinicians in the OR. Key activities of the ACT included (a) reviewing the anaesthetic plans for higher-risk cases and communicating recommendations to the OR anaesthesia clinicians, (b) continuous monitoring of real-time data and alerts to assess patient deterioration (c) encouraging adherence to institutional protocols (e.g., hyperglycaemia management), and (d) assisting in crisis management and coordinating out-of-OR resources. A pilot and feasibility trial of the ACT (ACTFAST-3) showed the feasibility of using a real-time telemedicine system for intraoperative decision support and collaborative decision-making.15 Additional design and process changes were made based on studies assessing the workflow of ORs and ACT 14 and patient input. 17
The objective of this clinical trial, Telemedicine Control Tower for the OR: Navigating Information, Care, and Safety (TECTONICS), was to evaluate the impact of remote monitoring and decision support by the ACT on postoperative patient outcomes.
Methods
Study design and ethics
TECTONICS was a pragmatic, randomised parallel, single-centre, superiority trial conducted at Barnes-Jewish Hospital (BJH) and Washington University School of Medicine in St. Louis, MO, USA. The institutional review board at Washington University approved the study with a waiver of informed consent. TECTONICS’s protocol and statistical analysis plan were registered at clinicaltrials.gov (NCT03923699). A data safety monitoring board (DSMB) met quarterly but conducted no interim efficacy analyses.
Setting and population
BJH is an adult urban tertiary centre. During the study (July 1, 2019 to January 30, 2023), 59 ORs were included, with at most 54 concurrent rooms. These excluded non-surgical procedure suites and obstetrics. The site used a medical-direction model for anaesthesia care: no more than 4, and usually 2-3, certified registered nurse anaesthetists (CRNA) or 1-2 resident physicians per anaesthesiologist. We enrolled all adult patients (>=18 years) with surgery starting between 06:15 and 16:00 weekdays with anaesthesia services, including emergency cases. The ACT was closed, and patients excluded on days with clinical staff shortages (see Supplement). Patients were followed for 30 days to ascertain outcomes.
Intervention: Anesthesiology Control Tower (ACT)
Patients in the intervention group received ACT-enhanced care; we have previously described the development, preliminary evaluation and functioning of the ACT. 14–16 The ACT was a remote monitoring suite with real-time data feeds from the Electronic Health Record (EHR) and the AlertWatch:OR (Ann Arbor, Michigan) informatics platform. ACT screens displayed a customized AlertWatch:OR interface with a dashboard summarizing all active surgeries. The dashboard contained an overall patient complexity measure, current data-driven alerts, and links to patient-specific views of current data and trends (Supplemental Figure 1). To assist with prioritizing cases for review and quantifying patient risks for ACT-OR communication, a web application displayed machine learning predictions of individual patient risk of several major adverse events (Supplemental Figure 2). 18–21 The implementation and validation of this web application is described elsewhere. 22 ACT staff also used the Epic EHR to access patient information.
The supplement contains the ACT manual of procedures. The ACT was staffed by an anaesthesiologist, a research coordinator, and up to 3 additional personnel (CRNA, student registered nurse anaesthetist, anaesthesiology resident). ACT activities focused on reviewing anaesthetic plans in active cases selected at the anaesthesiologist’s discretion, identifying important issues, and sending a summary of risk-mitigation recommendations and applicable protocols to the intraoperative anaesthesia clinicians. At their discretion, ACT clinicians reacted to alerts in the Alertwatch:OR dashboard, either sending recommendations to the intraoperative clinician or silencing the alert. The ACT contacted clinicians via Epic’s Secure Chat messaging or hospital-issued phones.
Staffing and monitoring were unchanged in the usual care group. ACT staff reviewed anaesthesia plans and alerts in usual-care patients, but the results were not communicated to intraoperative clinicians. ACT staff contacted intraoperative clinicians in usual-care ORs only in situations with potential imminent danger to patients, such as the failure to deliver inhaled anaesthetic agents.
Randomisation and Blinding
Each day, operating rooms (and all patients nested in each OR) were 1:1 randomised to intervention or usual care (cluster randomisation by operating room and day). Randomisation was calculated and displayed in the ACT dashboard at midnight without considering the number of cases scheduled in each room (including zero). Therefore, the realized randomisation ratio of patients varied. Only the first case for a patient in a 30-day window was analysed; cases 30-days after an index surgery were eligible for analysis. Outcome assessors and patients were blinded to assignments. Intraoperative clinicians were blinded unless contacted by the ACT.
Patient Outcomes
Four co-primary outcomes were ascertained: 30-day all-cause mortality, postoperative respiratory failure, postoperative acute kidney injury (AKI), and postoperative delirium. These outcomes were chosen to represent major patient-oriented complications. Secondary outcomes were a composite (sum of co-primary outcomes), hyperglycaemia, hypotension, normothermia, low peak airway pressures, avoidance of gaps in volatile anaesthetic delivery, and efficient use of volatile anaesthetics, which are operationalized in the supplement. Alternative specifications are also defined in the supplement. All outcomes were incident. Definitions, missing data handling, and EHR measurements used for primary and secondary outcomes are contained in the Supplement. Data was extracted from the EHR in two waves (2019-2020, 2021-2023).
Sample size calculation
Sample size was calculated per the protocol with a planned enrolment of 40,000 over 4 years. 23 As a low-risk health system intervention, there was no clear minimally clinically important difference. Due to a larger than expected number of eligible patients, during the first year of the study, the principal investigator and DSMB agreed to expand the study population to 80,000 to allow detection of smaller but potentially meaningful intervention effects.
Statistical analysis
Relative risks for all primary outcomes were calculated with a Poisson generalized estimating equation (GEE) model clustering on OR and day (the unit of randomisation) and HC1 adjustment to standard errors. 24 Risk differences were calculated with a linear-link GEE model using the same clustering. Bonferroni adjusted 95% and 99.5% confidence intervals were calculated. Two-sided p-values were adjusted for multiple testing 25 with an alpha of 0.005. 26 Secondary outcomes were analysed in the same manner; a linear link GEE model was used for non-binary outcomes. All analyses were conducted intention-to-treat (ignoring actual communication between the ACT and OR) among all patients with ascertainable outcomes. Planned subgroup and sensitivity analyses are described in the supplement. Analysis used R version 4.3.0; a docker file with package versions and analysis code is available at https://github.com/cryanking/tectonics_deident.
Intervention monitoring
ACT staff logged case reviews, alert interpretation, and OR communication (OR interventions) in the AlertWatch:OR web interface using discrete fields and free-text comments. Due to a database malfunction, ACT logs of case reviews, ACT to OR communication, and alert interpretations were lost for a block of 67/833 study days; we report contacts and alerts from discrete data on days with intact logs.
Results
Figure 1 displays the included population. The trial completed its targeted enrolment with 71927 surgeries included in the primary analysis, with 35956 cases (18091 clusters) allocated to intervention and 35971 cases (17907 clusters) allocated to usual care. There were minimal differences in comorbidities, functional status, surgery class, or other characteristics (Table 1). The mean patient age was 57 ranging from 18 to 103; patients were 49% male and 51% female. Patient races were white (70%) or Black (20%), with 9% unrecorded or belonging to multiple groups. A wide variety of surgery types were included; orthopaedics was the most common high-level grouping (18%).
Patient flow and exclusion criteria. ACT hours = 0615 to 1600. AKI =acute kidney injury. CAM-ICU = confusion assessment method for the ICU. “no flowsheet data” = no data returned for specified medical record number.
Baseline characteristics of patients. Mean (SD) or k/N (%),Groups with less than 100 patients not shown (merged into “Other”). Atrial Fib = atrial fibrillation, CAD = coronary artery disease, CKD = chronic kidney disease, ESRD = end stage renal disease, COPD = chronic obstructive pulmonary disease, VTE = venous thromboembolic disease, OSA = obstructive sleep apnoea, HTN = hypertension, METs = metabolic equivalents, ASA-PS = American Society of Anesthesiologists’ physical status, GI = gastroenterology. Patients with more than one included surgery appear multiple times.
Supplementary Figures 3, 4, and 5 show enrolment rates and ascertainment rates of primary and secondary outcomes over time. Aside from a period of reduced surgical volume and limited staff availability during the COVID-19 pandemic (Supplementary Figure 6), recruitment and outcome ascertainment were consistent and not differential by study group. The low rate of ascertainment of incident delirium was anticipated because routine CAM-ICU measurement was performed only in some ICUs. Intervention and usual care groups were well separated with respect to ACT-OR contact: the ACT contacted 2039/32224 (6.3%) of intervention cases and 238/31938 (0.7%) of usual care cases regarding alerts. ACT clinicians reviewed anaesthesia plans on 7977/32224 (25%) intervention cases and 5680/31938 (18%) usual care cases. Denominators for ACT actions are smaller than the total because of a database malfunction affecting several months.
For the co-primary outcomes, there was no significant difference in 30-day mortality [ACT 641/35956 (1.8%) vs usual care 638/35971 (1.8%), risk difference 0.0% (95% CI −0.2% to 0.3%), relative risk 1.01 (95% CI 0.87 to 1.16), p=0.96], respiratory failure [ACT 1089/34613 (3.1%) vs usual care 1112/34619 (3.2%), risk difference −0.1% (95% CI −0.4% to 0.3%), RR 0.98 (95% CI 0.88 to 1.09), p=0.96], AKI [ACT 2357/33897 (7%) vs usual care 2391/33795 (7.1%), risk difference −0.1% (−0.6% to 0.4%), RR 0.98 (95% CI 0.91 to 1.06), p=0.96], or delirium [ACT 1283/3928 (32.7%) vs usual care 1279/3989 (32.1%), risk difference 0.6% (−2.0% to 3.2%), RR 1.02 (95% CI 0.94 to 1.11), p=0.96]. Table 2 shows event rates and effect estimates for primary and secondary outcomes. Figure 2 shows time trends in unadjusted quarterly event rates in usual care and intervention groups.
Primary outcome rates by treatment group over time. Bars = pointwise 95% confidence intervals from a linear GEE model using the same clustering as the primary analysis and HC1 adjustment. Data aggregated over 3-month intervals. Red dots = ACT, black = usual care.
Intention-to-treat effect estimates of the intervention. Missing due to either outcome data not present, or patient not eligible, such as outcome present at baseline. Usual care and intervention columns [yes/number measured (%)] or [mean (SD)] for continuous outcomes. GEE coefficients from clustered Poisson model (binary outcome) or linear model (continuous outcomes). 95% CI Bonferroni corrected for multiple testing. P-value by permutation (5000 permutations) corrected for multiple testing. 25 Identical adjusted p-values due to step-down procedure; unadjusted results in Supplemental Table 9. AKI = acute kidney injury, PIP = peak inspiratory pressure. Secondary outcomes defined in the supplement.
After multiple testing correction, there were no significant differences by intervention group for any secondary outcome (Table 2). The results were consistent across alternative specifications, subgroup analysis, and sensitivity analyses (Supplement Tables 1-9).
Supplement Table 10 displays the frequency and reason for ACT interventions.
Discussion
This large (N=71927) randomised trial tested the hypothesis that compared to usual care, remote monitoring and telemedicine-based support from a novel Anesthesiology Control Tower using real-time alerts, machine learning risk identification, and in-depth case reviews for intraoperative anaesthesia clinicians would improve 4 key postoperative patient outcomes: 30-day mortality, respiratory failure, AKI, and delirium. We found no significant effects of randomisation to ACT support on any of the primary or secondary outcomes, nor did we find discordant results in subgroup or sensitivity analyses. Because mortality and respiratory failure are rare (Table 2) and multiple testing decreases certainty, confidence intervals on the effect sizes included some clinically meaningful values. However, for the more common outcomes (AKI and delirium) and secondary outcomes, the results excluded meaningful effect sizes (Table 2, Supplemental Table 9).
Comparison to prior trials
The only directly comparable study for synthesis is our pilot trial, ACTFAST-3, 15 which had similar findings regarding effects of the ACT on patient outcomes and process measures. Notably, ACTFAST-3 was smaller and designed to detect changes in process measures (temperature and glycaemic control). TECTONICS is also distinguished from ACTFAST-3 by having a stable intervention versus adaptation during implementation and the deployment of a machine learning risk-estimation tool. Our results should also be compared to Kheterpal et al., who quasi-experimentally studied the impact of the Alertwatch:OR application for intraoperative clinicians on anaesthesia process measures, finding that time with hypotension, inappropriately large tidal volumes, and costs improved. 13 Our results are therefore surprising given the most serious alerts were communicated to the intraoperative clinician, and that very few clinicians at the study site otherwise used Alertwatch:OR.
Our intervention is an adaptation of widely adopted strategies in tele-critical care. 4 Our results are therefore surprising given the positive association of TCC with ICU patient outcomes.6 The study setting, an urban academic centre, also mirrors the settings in which TCC was most associated with improved patient outcomes. 6 However, there are no individual-level RCTs of TCC’s effect on patient outcomes, and a variety of biases may have led to over-estimation of effects in the before-after studies which dominate tele-critical care’s evidence base. A cluster-randomised trial of TCC with a lower risk of bias will be available soon. 27 The only randomised trial of TCC of which we are aware found effects on protocol adherence but not patient outcomes. 5 Importantly, because of the (nearly) patient level randomisation and contamination from shared clinicians, our analysis of TECTONICS does not capture benefits to outcomes due to overall improvements in protocol adherence, discussed below.
Strengths of TECTONICS include the low crossover rate, large and diverse population, low loss to follow up, and pragmatic design. TECTONICS was preceded by iterative development and a substantial pilot, making the intervention relatively mature. Because of the long period of ACTFAST-3 and TECTONICS (2017-2023), our analysis avoids threats to generalization from sustainability and acceptance from clinical stakeholders.
Several limitations could have led to underestimating the effects of the ACT. Although the population was large, the rate of contact between the ACT and eligible ORs was moderate (6.3%), so even a highly efficacious intervention (conditional on contact) would have had modest power. An exploratory analysis of only patients whom the ACT was likely to contact and other sources of heterogeneity is a future direction, but our planned subgroup analyses found no effect of the intervention. Hawthorne and contamination effects are also limitations. Clinicians at the study site were aware of ACT monitoring and received frequent messages regarding common issues. Although specific physiologic alerts did not affect usual care patients, the “audit and feedback” type effect on protocol adherence and optimization of common intraoperative issues likely spilled over, affecting both study arms. For example, an anaesthesiologist caring for diabetic patients in both arms of the study might improve glycaemia protocol adherence in both groups based on ACT messages. We initially planned a pre-post intervention analysis to elucidate these effects, but changes in the EHR and the concurrent COVID-19 pandemic made that infeasible. Secondary analyses will explore changes after the study to address this question. Although the recorded crossover’ rate was low, the primary analysis likely does not reflect the effect of the most clinically-relevant interventions made by the ACT, because the study encouraged clinicians to intervene on usual care-group patients if serious time-sensitive alerts occurred (Supplemental Table 10). Finally, measurement error may have mitigated observed effects of the intervention. Two of our primary outcomes, delirium and respiratory failure, were derived from routinely collected bedside assessments, and inaccuracies 28 in those data may have distorted our results. Deaths were identified from the EHR rather than governmental statistics and may miss some events after discharge if no follow-up was planned. AKI is likely under-detected due to selective measurement of postoperative creatinine. In none of these cases do we believe these errors to be differential between groups.
The study site was a tertiary academic centre, which limits the generalizability of the findings. Several aspects of the study site may explain the null result. The site’s low supervision ratio allowed anaesthesiologists to closely monitor high-risk cases, reducing the impact of cognitive load and the opportunity to miss key data. Anaesthesia clinicians at the site sub-specialize, which tends to make them familiar with relevant protocols and best practices, 29 mitigating the relevance of the ACT’s recommendations. The site’s pre-anaesthesia clinic evaluated roughly 90% of patients, pre-emptively facilitating appropriate risk mitigation plans. 19 In settings without these redundant checks, a similar intervention may improve outcomes. Additionally, we evaluated a single set of physiologic alerts. The most common alerts were on topics only weakly related to the primary outcomes (Supplemental Table 10). Although we found these alerts to fit the needs of the ACT, alerts with more advanced detection of, for example, impending hypotension, may have larger effects.
Summary
In this large RCT, we found that intraoperative telemedicine support for anaesthesia clinicians with case planning reviews and real-time alerts did not reduce 30-day mortality, respiratory failure, AKI, or delirium. Clinicians accepted telemedicine support, and we observed many safety and process-of-care interventions during its development and implementation, which suggests that intraoperative telemedicine should be further explored in other settings. Future work on similar telemedicine models should use multi-centre designs to detect effects mediated by quality improvement, avoid contamination due to time-critical alerts that cannot ethically be withheld from the intraoperative clinician, and explore alternative practice settings.
Data Availability
The Washington University Human Research Protection Office did not permit sharing of patient level data due to enrolment with a waiver of consent. Summary data is available at the clinicaltrials.gov registration.
Author Contributions
All authors had full access to all the data in the study and agreed to submit it for publication. All authors contributed to the revision of the manuscript. CRK wrote the manuscript draft, processed the EHR data, and did the statistical analysis. CRK, BAF, ST, and MA implemented the machine-learning web application. ABA created the original analysis plan, contributed to the conception of the work, and analysed data for and prepared DSMB reports. CRK and ABA directly accessed and verified the underlying patient data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. BAF and CRK directly accessed and verified the case-review, and alert data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. SHG, BT, OH, PK, BH, TSW, MCP, JA, and MSA contributed to the conception of the work, implementation of the intervention, and interpretation of the results. TB contributed to the implementation of the intervention, data acquisition, and quality control of case-reviews and alert data. SM contributed to the implementation of the intervention and regulatory oversight. SG, AM, DM, MAV, and EK contributed to the implementation of the intervention and data acquisition. AK conducted EHR data acquisition and intervention informatics implementation. MSA lead the conception, design, and implementation of the work. MSA, TK, BAF, ABA, and CRK contributed to the interpretation of the data.
Declaration of interests
We declare no competing interests.
Funding
This work was supported by the National Institute of Nursing Research (R01 NR017916 to Dr. Avidan) and departmental funding from Washington University in St Louis School of Medicine. The investigators were also supported by National Institutes of Health training awards TR002346 (Dr King) and T32GM108539 (Drs King and Fritz) and funding from the Foundation for Anesthesia Education and Research (MRT08152020 to Dr Fritz).
The funding organization(s) had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.
Data Sharing Statement
The Washington University Human Research Protection Office did not permit sharing of patient level data due to enrolment with a waiver of consent. Summary data is available at the clinicaltrials.gov registration.
Non-author collaborators: the ACTFAST author group
Acknowledgements
Members of the ACTFAST Study Group who contributed to data acquisition or other support of the implementation are provided in the Supplement.
REFERENCES
