Quantifying and Visualizing Emergency Physician Workflow: Results of an Observational Time-Motion Study
=======================================================================================================

* Andrew J. Henreid
* Kimon L.H. Ioannides
* Joshua M. Pevnick
* Tara N. Cohen
* Sam S. Torbati
* Teryl K. Nuckols
* Carl T. Berdahl

## ABSTRACT

**Background** Emergency physicians face considerable workflow challenges due to unpredictable work environments, frequent interruptions, and mounting documentation requirements. Excessive time away from direct patient care is increasingly viewed as detrimental to care quality, communication, and patient safety. This study aimed to quantify and visualize the time emergency physicians spend on specific activities during their clinical shifts, particularly computer usage.

**Methods** This observational time-motion study was conducted in a high-volume, urban emergency department (ED). An observer used a web-based application to track physician activities including computer use, direct patient care, and all other major tasks carried out on shift. Electronic health record (EHR) event log data was queried to measure computer use after each physician’s scheduled shift. The primary outcome was total minutes of computer time (during and after shift) per scheduled hour of clinical work.

**Results** The observer tracked 20 emergency physicians for one 8-9h clinical shift each, which generated 150.0 hours of real-time observation data quantifying physicians’ ED workflow. In total, emergency physicians spent a median 29.8 minutes (IQR 25.6-38.5) on the computer per scheduled hour of their ED shift. Physicians spent a median 34.1% of their shift time using the computer and 26.9% with patients. Other activities included verbal communication with staff (15.9%), phone use (9.5%), miscellaneous tasks (5.5%), personal time (3.9%), electrocardiogram review (0.7%), and procedures (0.4%). EHR log analysis showed that physicians spent an additional median 1.3h (IQR 0.5-2.6) using the computer after their scheduled shifts.

**Conclusion** Emergency physicians spent more than one-third of their ED shift working on the computer, which was more time than they spent with patients. They also spent 1-2 hours using the computer after their shifts. These findings demonstrate the need for strategies aimed at reducing unnecessary computer use during and after clinical shifts to enhance efficiency and improve patient care.

Keywords
*   Emergency Medicine
*   Workflow
*   Physicians
*   Electronic Health Records
*   Computers
*   Patient Care
*   Documentation
*   Efficiency
*   Time Management
*   Time and Motion Studies
*   Technology
*   Working Conditions
*   Clinical Decision-Making
*   Health Services Research

## INTRODUCTION

### Background

There is rising concern that too much time spent away from the bedside may undermine quality of medical care due to decreased communication and fewer opportunities to directly interact with patients.1 Recent studies of physician workflow in the United States and internationally have concluded that frequent interruptions and burdensome documentation requirements place patients at risk for harm.2,3 The emergency department (ED) setting is a particularly challenging work environment for physicians given its inherent unpredictability. Prior literature has demonstrated that interruptions in the ED are persistent, and that task switching can jeopardize patient safety.4

The application of methods from human factors engineering, such as time-motion studies, can help identify inefficiencies, optimize performance, and improve work design.5-7 In preparation for tests exploring the impact of tools designed to reduce time at the computer in the ED, we sought to establish baseline measurements.

### Objectives

The objective of this study was to assess and characterize emergency physician workflow by quantifying the time ED physicians spend on pre-specified tasks during their clinical shifts. The primary outcome was the total minutes of computer time per scheduled hour of clinical shift. Secondary outcomes included the percentage of time spent on specific activities, including using the computer, interacting with patients, verbal communication with clinical staff, operating the phone, reviewing electrocardiograms (EKG), performing medical procedures, personal time, and all other nonspecific miscellaneous tasks.

## METHODS

### Design and Setting

We conducted an observational time-motion study and review of EHR event logs to track emergency physician activities in an urban, high-volume ED that is a Comprehensive Stroke Center, ST Elevation Myocardial Infarction-Receiving Center, and Level 1 Trauma Center with an annual census of 90,000 ED visits. Observations were conducted between May 21, 2019, and November 6, 2019. During the study period, attending emergency physicians supervised or provided care for all patients. Residents from non-emergency medicine specialties (e.g., internal medicine, anesthesia, etc.) assisted with care for less than 5% of all visits, and physician assistants (PAs) were involved with 20%. The number of emergency physicians staffing the ED varied from a minimum of 3 during overnight shifts to a maximum of 6 during peak hours.

### Participants

Participant eligibility criteria included emergency physicians credentialed to practice in the institution’s ED. No other exclusion criteria were used for this study sample. We obtained written informed consent from 20 attending emergency department physicians. Physicians were recruited through convenience sampling and consented to the study in a private setting prior to their observed shift. The institutional review board approved a waiver of consent for patients, as the primary focus of the study was to observe physician activity where contact with patients and sensitive patient information was expected to occur only incidentally.

### Variables

The primary outcome of interest was the total minutes of computer time per scheduled hour of clinical shift. Calculation of total computer time included the number of minutes each physician spent on the computer, both during and immediately after their scheduled ED shift. Secondary outcomes of interest were the percentage of time spent on specific clinical activities, including computer usage (Computer), direct patient interaction (Patient), verbal communication with clinical staff (Verbal), operating the phone (Phone), reviewing EKGs (EKG), and performing procedures (Procedures).

EHR event log data provided detailed timestamps for all user actions carried out within the EHR computer system (e.g., “ED Disposition Activity viewed,” “User logged in,” “Notes viewed,” etc.) after each observed shift. *EHR use* was defined as the number of seconds each physician spent *actively* using the EHR during their event log timespan, using a 1 minute threshold of inactivity.8 Physician EHR use was thus considered inactive if more than 59 seconds elapsed between any two successive EHR actions, indicating the physician left the workstation or diverted their attention to perform other tasks.8,9 *Total active use* was determined by calculating the time intervals between access times (access instants) for every active EHR action (i.e. the number of seconds between every two subsequent logged events).

*Pajama Time* is a term attributed in academic literature to characterize physician time spent using the EHR (computer) after the end of a scheduled shift (i.e., on administrative actions such as writing clinical notes).10-12 In this study, we defined pajama time as any EHR use that occurred within two days after in-person observation concluded or before the physician’s next clinical shift.10,11,13 Pajama time was measured using EHR event logs capturing EHR actions up to 48 hours after the observed shift day (two-day span), which included EHR use occurring either on-site at the medical center or elsewhere (i.e., at home). ***See* Figure 1*. Overview of Data Collection Process*** for a depiction of the combined data collection methods utilized in this study.

![Figure 1.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/12/01/2024.11.27.24318109/F1.medium.gif)

[Figure 1.](http://medrxiv.org/content/early/2024/12/01/2024.11.27.24318109/F1)

Figure 1. Overview of Data Collection Process

### Data Sources

#### Time-motion observation data

In-person, time-motion observations were carried out to collect workflow data characterizing ED physician time spent on pre-specified tasks during ED shifts (one observed shift per enrolled physician). A single, trained observer with experience in clinical care as an emergency medical technician (AJH) shadowed enrolled emergency physicians during clinical shifts that were scheduled to last 8 or 9 hours. The observer began data collection at the time the shift began and ended only when the physician informed the observer that they were done interacting with patients and planned to either leave the ED or use the computer without further patient interaction. The observer took a 45-minute break from data collection approximately halfway into each shift and length of shift was adjusted to account for this period in the primary outcome analysis.

The trained observer used a digital tablet and web-based, time-motion application (*TimeCaT 3.9*) to track start and end times (to the nearest second) across a list of pre-specified tasks.14 Time-motion studies are a well-established methodology for workflow research and are considered “gold standard methods” for healthcare observations.6 Similarly, the TimeCaT application used for in-person data collection was specifically designed and validated for measuring workflow in healthcare contexts.14 ***See* Figure 2*. TimeCaT Data Capture Interface*** for the graphical user interface used for in-person observation data collection.

![Figure 2.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/12/01/2024.11.27.24318109/F2.medium.gif)

[Figure 2.](http://medrxiv.org/content/early/2024/12/01/2024.11.27.24318109/F2)

Figure 2. TimeCaT Data Capture Interface (Time-Motion)

Physician workflow (defined as the series of activities carried out by the physician during their shift)15 was continuously assigned to one primary task by clicking an activity-specific button in the TimeCaT application. These tasks included computer use, direct patient interaction, variations of phone use, EKG review, other clinical procedures, verbal communication with hospital staff, verbal instruction to staff, reading or writing on paper, waiting for next task, gathering equipment, walking within the medical center, personal time, and miscellaneous actions. For clarity and comprehension, individual tasks were then combined into the following activity categories: computer use (Computer), patient interaction (Patient), verbal communication with ED staff (Verbal), operating the phone (Phone), EKG review (EKG), clinical procedures (Procedures), all personal tasks (Personal), and any remaining miscellaneous tasks (Miscellaneous). ***See Table 1. Categorization of Physician Activities*** for details on how individual tasks were classified into major activities.

View this table:
[Table 1.](http://medrxiv.org/content/early/2024/12/01/2024.11.27.24318109/T1)

Table 1. Categorization of Physician Activities (Time-Motion)

When physicians were performing multiple tasks simultaneously (e.g., talking on the phone while using the computer), the observer made a determination about which task was the primary focus of the physician’s attention. This, in addition to the highly variable nature of ED workflow, resulted in many transitions between tasks during periods of multi-tasking, which the TimeCaT software was designed to capture.14 When the physician entered patient rooms, the observer waited outside the room in the interest of patient privacy, and time spent inside the patient’s room was assumed to be spent on direct patient care.

### Event log data from the electronic health record

To supplement the time-motion data collected during in-person observations, we also obtained electronic health record (EHR) event log data to quantify the amount of time physicians spent on the EHR after the end of each physician’s observed shift.9,16,17 Each event log contained a detailed list of actions performed within the EHR by a particular user over a specified period of time, and each action is accompanied by a time-stamp.16-19 We collected the EHR event log data from the time in-person observation ended until up to 48 hours later or the time that the physician’s next shift began (whichever was earlier). One physician’s EHR event log data was excluded from “pajama time” analysis because their unique administrative responsibilities suggested that their EHR actions were not directly related to their clinical care.

### Statistical Methods

Descriptive analyses were performed using Microsoft Excel (Version 2402) and SPSS Statistics (Version 24.0). Data visualizations were created using Tableau Desktop (Version 2024.2), Microsoft Office, and Seaborn (Version 2021). Data visualizations were used assess physician workflow allocation by activity type and sequence of activities over the course of each ED shift.

### Patient and Public Involvement

Patients and the public were not involved in the design of this study. Research objectives were developed by the study team based on existing gaps in understanding around ED workflow. The results of this study will be disseminated through academic publications targeting healthcare professionals and institutional stakeholders. Findings may inform strategies to improve ED efficiency, which could benefit patient care experiences in the future.

## RESULTS

### Overview

Twenty attending emergency physicians were enrolled in the study (5 female, 15 male). The median number of years in practice since residency graduation was 14.5 (IQR 7.5-23.0; range 3 to 44 years), with a range in residency graduation year of 1976 to 2017. We collected a total of 150.0 hours of time-motion data, with a median observation duration of 7.3h (IQR 7.2-8.1) per physician. This included a total of 14 eight-hour shifts and 6 nine-hour shifts with a shift start time range of 5:00 AM–1:00 PM.

EHR event log data were obtained for all 20 clinical shifts, starting from the time the shift observation ended and continuing for up to 48 hours or until the beginning of the physician’s next clinical shift. Initially, our dataset of EHR event logs included descriptions of physician actions over a total of 200.3h including pajama time. After exclusion of one physician with unique administrative responsibilities, EHR event log data used in the reported analysis included a total of 189.1h [median 10.1h (IQR 8.5-10.6) per physician] across the remaining 19 physicians in the sample.

### Analysis

For the primary outcome variable of total minutes of computer time per scheduled hour of clinical shift, we found that ED physicians spent a median 29.8 minutes (IQR 25.6-38.5) on the computer per each hour of scheduled clinical work. Total minutes spent on the computer included directly observed computer time during each shift in addition to EHR pajama time that occurred after the physician’s observed ED shift.

#### Direct Observation of ED Workflow

Based on real-time observational data across all 20 ED shifts, emergency physicians spent a median 34.1% of their shift on the computer [156.5 minutes (IQR 145.2-179.3)]. The longest interval of time away from the computer at any point during a shift was 20.0 minutes (range 8.0 to 20.0). Secondary to computer use, physicians spent a median of 26.9% of their shift with patients [115.2 minutes (IQR 102.0-154.1)] Among all 20 physicians, the longest time continuously spent at a patient’s bedside for any physician was 21m 47s (range 7.4 to 21.8 minutes).

Additionally, ED physicians spent a median of 15.9% of their shift [73.3 minutes (IQR 45.6-96.4)] verbally communicating with staff. Physicians spent a median 9.5% of their total shift using the phone to talk to other healthcare workers [41.4 minutes (IQR 33.4-59.2)], with a median duration of 1m 5s per phone task. Miscellaneous clinical tasks consisted of a median 5.5% [24.2 minutes (IQR 18.4-30.1)] of their shift. Clinical procedures included 0.7% EKG review [median 3.8 minutes (IQR 1.4-5.0)] and 0.4% other procedures [median 1.7 minutes (IQR 0.6-6.4)]. Time spent on EKG review is described in more detail in a related manuscript that focused on interruptions.20 Across all physicians, personal time taken while working constituted only 3.9% of their total shift workflow [median 16.8 minutes (IQR 9.9-25.4)] and included time spent eating, using the restroom, and personal smartphone use.

Visualizations of physician workflow illustrate the substantial allocation of time spent on the computer compared to other clinical priorities as well as variations in workflow between and throughout ED shifts. ***See Figures 3-5***. for visual depictions of how observed physicians’ time was spent on shift.

**Figures 3-5 visualize time spent by activity across all physicians (Figure 3), by individual physician shift (Figure 4), and linearly from start to finish for each shift (Figure 5)**

![Figure 3.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/12/01/2024.11.27.24318109/F3.medium.gif)

[Figure 3.](http://medrxiv.org/content/early/2024/12/01/2024.11.27.24318109/F3)

Figure 3. Allocation of Physician Time in the ED (Percent of Total Shift Across 20 Observed Physicians)

![Figure 4.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/12/01/2024.11.27.24318109/F4.medium.gif)

[Figure 4.](http://medrxiv.org/content/early/2024/12/01/2024.11.27.24318109/F4)

Figure 4. Allocation of Physician Time in the ED (By Physician)

![Figure 5.](http://medrxiv.org/https://www.medrxiv.org/content/medrxiv/early/2024/12/01/2024.11.27.24318109/F5.medium.gif)

[Figure 5.](http://medrxiv.org/content/early/2024/12/01/2024.11.27.24318109/F5)

Figure 5. Timelines of Physician Workflow in the ED (By Physician)

#### After-Shift EHR Activity

Throughout 19 clinical shifts, we obtained data on 112,396 EHR log events (actions) [median 5732 (IQR 5213-6519)] with a median event rate of 9.9 events per minute. The total EHR log timespan recorded from the log data, including active and idle EHR use, was approximately 189.1 hours across 19 included physicians. Physicians spent a median 1.3 hours [(IQR 3.1-5.0); range 1.9h to 9.2h] for pajama time after the end of their scheduled shift.

## DISCUSSION

In this study of emergency physician workflow in a high-volume, metropolitan, Level 1 trauma center, our results reveal that physicians spent more time using computers than engaging in direct patient care — generally spending more than a third of their shift on the computer or approximately half of every scheduled hour when accounting for pajama time after their shift.

These findings add to a growing body of evidence that time spent on direct patient care is limited by competing workflow demands (especially burden of computer use) associated with emergency department care delivery.21-24

This study combines data from both in-person time-motion observations and EHR event log data to demonstrate that the burden of computer use is not limited to time on-shift but also commonly extends hours beyond the end of clinical shifts.10,12,16 While EHRs facilitate improved patient care by providing physicians access to patients’ prior medical histories and test results, for example, they also complicate the clinical work environment and may lead to unintended consequences such as limiting physician time at patients’ bedsides.25,26

Emergency physicians are required to adhere to documentation requirements, including written medical histories, physical examinations, interpretations of diagnostic test results, and medical decision-making notes.27 While some documentation requirements have recently been revised to make them less burdensome, emergency physicians are still required to document clinical risk and complexity for billing purposes, which detracts from time available for direct patient care.28,29

Although current EHR systems facilitate access to essential clinical information, their user interface and design limitations may place an additional cognitive burden on emergency physicians.30-32 Overly complicated interfaces and unintuitive action sequences in EHRs may frustrate physicians and place patients at risk for medical errors.33-35 Pragmatic user-centered design principles should be incorporated as EHR systems are designed and tested, especially for those employed in ED care settings where time-critical actions are commonplace.

The development of more user-friendly interfaces that support seamless compliance and mitigate risk could significantly alleviate existing administrative pressures and allow physicians to focus more on patient-centered care.35,36 Developing these systems requires additional research to understand how clinicians’ varied current and future approaches to EHR use may impact the quality of care, for example by assessing whether more “staccato” (frequently interrupted) or, conversely, “legato” (fewer, longer periods of use) interactions are associated with increased efficiency or errors.

There is an urgent need for improved EHR design, documentation processes, and other human factors-centered solutions that enhance efficiency and improve quality and safety for patients.37,38 Investing in “elbow-to-elbow” EHR training for emergency physicians may improve understanding of EHR functionalities, thereby enhancing their efficiency and reducing the associated burden for current and future clinicians.39,40

Another emerging solution is the implementation of “ambient artificial intelligence scribes,” which use natural language processing to transform real-time audio-recordings of clinical encounters into a clinical note.41 This technology is more promising than ever before due to rapid advancements in artificial intelligence (AI) capabilities over the last several years. However, careful evaluation of such products must be undertaken to understand the strengths and limitations of the technology and identify unintended consequences for physicians (i.e., unexpected workflow challenges) as well as patients.42 A recent article from a team at Kaiser Permanente expressed early enthusiasm for the use of ambient AI scribes, but they did note that the tool was not well-integrated into their EHR at the time of testing, which led to technical difficulties and frustrations due to wasted time.41 Our study lays the groundwork for testing of tools such as AI scribes at our own site, since we now have baseline time-motion data that can be compared to future time-motion data collected during future tests of change utilizing innovative EHR and documentation tools.

### Limitations

Our study has several limitations. First, this was a single-site study, which may limit the generalizability of our results. Second, we recruited a convenience sample of attending physicians to participate in the study. As such, it is possible that physicians who opted to participate have systematic differences in the way they spend their time compared to those who did not participate. Third, because our observer was instructed not to enter patients’ rooms due to privacy concerns, we assumed that time in rooms was entirely spent on direct patient care. Thus, it is possible that we overestimated the amount of direct patient care and underestimated the amount of time spent on other activities such as computer use and phone calls that could have occurred within patient rooms. Fourth, to incorporate EHR event log data into our analyses, we selected inactivity period cutoffs times based on qualitative comparisons of EHR event log data and in-person observation data that nonetheless may systematically over- or underestimate EHR time; however, similar inactivity windows have been used for EHR event log data in existing literature.8,9 Lastly, some physicians may have been using pajama time to catch up on documentation from prior shifts, which could have led us to overestimate total computer time.

### Conclusion

This time-motion study found that emergency physicians spend more than one-third of their time on the computer during their ED shifts. We analyzed EHR event logs to determine these physicians also spend an additional 1-2 hours using computers after the end of their scheduled shift. In total, emergency physicians spend approximately half of every scheduled shift hour on computer-based work. Future research should investigate strategies to minimize low-yield time at the computer and increase time spent providing direct patient care.

### Competing Interest Statement

No competing interests to declare.

## Data Availability

All data produced in the present study are available upon reasonable request to the authors

*   Received November 27, 2024.
*   Revision received November 27, 2024.
*   Accepted December 1, 2024.


*   © 2024, Posted by Cold Spring Harbor Laboratory

The copyright holder for this pre-print is the author. All rights reserved. The material may not be redistributed, re-used or adapted without the author's permission.

## References

1.  1.Alami H, Lehoux P, Gagnon MP, et al. Rethinking the electronic health record through the quadruple aim: time to align its value with the health system. BMC Med Inform Decis Mak 2020;20(1):32. doi: 10.1186/s12911-020-1048-9
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1186/s12911-020-1048-9&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32066432&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

2.  2.Füchtbauer LM, Nørgaard B, Mogensen CB. Emergency department physicians spend only 25% of their working time on direct patient care. Dan Med J 2013;60(1):A4558.
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=23340186&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

3.  3.Schneider A, Williams DJ, Kalynych C, et al. Physicians’ and nurses’ work time allocation and workflow interruptions in emergency departments: a comparative time-motion study across two countries. Emerg Med J 2021;38(4):263–68. doi: 10.1136/emermed-2019-208508
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzOC80LzI2MyI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzEyLzAxLzIwMjQuMTEuMjcuMjQzMTgxMDkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

4.  4.Werner NE, Holden RJ. Interruptions in the wild: Development of a sociotechnical systems model of interruptions in the emergency department through a systematic review. Appl Ergon 2015;51:244–54. doi: 10.1016/j.apergo.2015.05.010
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.apergo.2015.05.010&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=26154223&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

5.  5.Chisholm CD, Collison EK, Nelson DR, et al. Emergency department workplace interruptions: are emergency physicians “interrupt-driven” and “multitasking”? Acad Emerg Med 2000;7(11):1239–43. doi: 10.1111/j.1553-2712.2000.tb00469.x
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1553-2712.2000.tb00469.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=11073472&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000167382500013&link_type=ISI) 

6.  6.Lopetegui M, Yen PY, Lai A, et al. Time motion studies in healthcare: what are we talking about? J Biomed Inform 2014;49:292–9. doi: 10.1016/j.jbi.2014.02.017
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jbi.2014.02.017&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=24607863&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

7.  7.Kalne PS, Mehendale AM. The Purpose of Time-Motion Studies (TMSs) in Healthcare: A Literature Review. Cureus 2022;14(10):e29869. doi: 10.7759/cureus.29869
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7759/cureus.29869&link_type=DOI) 

8.  8.Sinha A, Stevens LA, Su F, et al. Measuring Electronic Health Record Use in the Pediatric ICU Using Audit-Logs and Screen Recordings. Appl Clin Inform 2021;12(4):737–44. doi: 10.1055/s-0041-1733851
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1055/s-0041-1733851&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=34380167&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

9.  9.Wang JK, Ouyang D, Hom J, et al. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One 2019;14(2):e0205379. doi: 10.1371/journal.pone.0205379
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0205379&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30726208&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

10. 10.Saag HS, Shah K, Jones SA, et al. Pajama Time: Working After Work in the Electronic Health Record. J Gen Intern Med 2019;34(9):1695–96. doi: 10.1007/s11606-019-05055-x
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1007/s11606-019-05055-x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31073856&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

11. 11.Dyrbye LN, Gordon J, O’Horo J, et al. Relationships Between EHR-Based Audit Log Data and Physician Burnout and Clinical Practice Process Measures. Mayo Clin Proc 2023;98(3):398–409. doi: 10.1016/j.mayocp.2022.10.027
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.mayocp.2022.10.027&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=36868747&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

12. 12.Rotenstein LS, Holmgren AJ, Horn DM, et al. System-Level Factors and Time Spent on Electronic Health Records by Primary Care Physicians. JAMA Netw Open 2023;6(11):e2344713. doi: 10.1001/jamanetworkopen.2023.44713
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2023.44713&link_type=DOI) 

13. 13.Magon HS, Helkey D, Shanafelt T, et al. Creating Conversion Factors from EHR Event Log Data: A Comparison of Investigator-Derived and Vendor-Derived Metrics for Primary Care Physicians. AMIA Annu Symp Proc, 2023:1115–24.
    
    

14. 14.Lopetegui M, Yen PY, Lai AM, et al. Time Capture Tool (TimeCaT): development of a comprehensive application to support data capture for Time Motion Studies. AMIA Annu Symp Proc 2012;2012:596–605.
    
    

15. 15.Davis MM, Gunn R, Cifuentes M, et al. Clinical Workflows and the Associated Tasks and Behaviors to Support Delivery of Integrated Behavioral Health and Primary Care. J Ambul Care Manage 2019;42(1):51–65. doi: 10.1097/JAC.0000000000000257
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/JAC.0000000000000257&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30499901&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

16. 16.Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: Primary Care Physician Workload Assessment Using EHR Event Log Data and Time-Motion Observations. Ann Fam Med 2017;15(5):419–26. doi: 10.1370/afm.2121
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTUvNS80MTkiO3M6NDoiYXRvbSI7czo1MDoiL21lZHJ4aXYvZWFybHkvMjAyNC8xMi8wMS8yMDI0LjExLjI3LjI0MzE4MTA5LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

17. 17.Rule A, Melnick ER, Apathy NC. Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures. J Am Med Inform Assoc 2022;30(1):144–54. doi: 10.1093/jamia/ocac177
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jamia/ocac177&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=36173361&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

18. 18.Burden M, Keniston A, Pell J, et al. Unlocking inpatient workload insights with electronic health record event logs. J Hosp Med 2024;n/a(n/a) doi: 10.1002/jhm.13386
    
    

19. 19.Adler-Milstein J, Adelman JS, Tai-Seale M, et al. EHR audit logs: A new goldmine for health services research? J Biomed Inform 2020;101:103343. doi: 10.1016/j.jbi.2019.103343
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jbi.2019.103343&link_type=DOI) 

20. 20.Ioannides KLH, Brownstein DJ, Henreid AJ, et al. Quantifying Emergency Physician Interruptions due to Electrocardiogram Review. J Emerg Med 2021;60(4):444–50. doi: 10.1016/j.jemermed.2020.11.024
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.jemermed.2020.11.024&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33414047&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

21. 21.Sinsky C, Colligan L, Li L, et al. Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties. Ann Intern Med 2016;165(11):753–60. doi: 10.7326/M16-0961
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/M16-0961&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=27595430&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

22. 22.Morley C, Unwin M, Peterson GM, et al. Emergency department crowding: A systematic review of causes, consequences and solutions. PLoS One 2018;13(8):e0203316. doi: 10.1371/journal.pone.0203316
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1371/journal.pone.0203316&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

23. 23.Konrad TR, Link CL, Shackelton RJ, et al. It’s about time: physicians’ perceptions of time constraints in primary care medical practice in three national healthcare systems. Med Care 2010;48(2):95–100. doi: 10.1097/MLR.0b013e3181c12e6a
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/MLR.0b013e3181c12e6a&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=20057331&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000274081900003&link_type=ISI) 

24. 24.Braddock CH, Snyder L. The doctor will see you shortly. The ethical significance of time for the patient-physician relationship. J Gen Intern Med 2005;20(11):1057–62. doi: 10.1111/j.1525-1497.2005.00217.x
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1111/j.1525-1497.2005.00217.x&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=16307634&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000232892800015&link_type=ISI) 

25. 25.Menachemi N, Collum TH. Benefits and drawbacks of electronic health record systems. Risk Manag Healthc Policy 2011;4(null):47–55. doi: 10.2147/RMHP.S12985
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.2147/RMHP.S12985&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=22312227&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

26. 26.Dymek C, Kim B, Melton GB, et al. Building the evidence-base to reduce electronic health record-related clinician burden. J Am Med Inform Assoc 2021;28(5):1057–61. doi: 10.1093/jamia/ocaa238
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jamia/ocaa238&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33340326&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

27. 27.Berdahl CT, Moran GJ, McBride O, et al. Concordance Between Electronic Clinical Documentation and Physicians’ Observed Behavior. JAMA Netw Open 2019;2(9):e1911390. doi: 10.1001/jamanetworkopen.2019.11390
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2019.11390&link_type=DOI) 

28. 28.Momenipur A, Pennathur PR. Balancing Documentation and Direct Patient Care Activities: A Study of a Mature Electronic Health Record System. Int J Ind Ergon 2019;72:338–46. doi: 10.1016/j.ergon.2019.06.012
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.ergon.2019.06.012&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32201437&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

29. 29.Erickson SM, Rockwern B, Koltov M, et al. Putting Patients First by Reducing Administrative Tasks in Health Care: A Position Paper of the American College of Physicians. Ann Intern Med 2017;166(9):659–61. doi: 10.7326/M16-2697
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/m16-2697&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

30. 30.Ahmed A, Chandra S, Herasevich V, et al. The effect of two different electronic health record user interfaces on intensive care provider task load, errors of cognition, and performance. Crit Care Med 2011;39(7):1626–34. doi: 10.1097/CCM.0b013e31821858a0
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1097/CCM.0b013e31821858a0&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=21478739&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 
    
    [Web of Science](http://medrxiv.org/lookup/external-ref?access_num=000291721800004&link_type=ISI) 

31. 31.Rotenstein LS, Molina M. Exploration of Electronic Health Record Patterns of Emergency Physicians-Charting the Digital Burden. JAMA Netw Open 2024;7(8):e2429749. doi: 10.1001/jamanetworkopen.2024.29749
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2024.29749&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=39136950&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

32. 32.Clarke MA, Schuetzler RM, Windle JR, et al. Usability and cognitive load in the design of a personal health record. Health Policy and Technology 2020;9(2):218–24. doi: 10.1016/j.hlpt.2019.10.002
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.hlpt.2019.10.002&link_type=DOI) 

33. 33.Mazur LM, Mosaly PR, Moore C, et al. Association of the Usability of Electronic Health Records With Cognitive Workload and Performance Levels Among Physicians. JAMA Netw Open 2019;2(4):e191709. doi: 10.1001/jamanetworkopen.2019.1709
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2019.1709&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=30951160&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

34. 34.Kroth PJ, Morioka-Douglas N, Veres S, et al. Association of Electronic Health Record Design and Use Factors With Clinician Stress and Burnout. JAMA Netw Open 2019;2(8):e199609. doi: 10.1001/jamanetworkopen.2019.9609
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1001/jamanetworkopen.2019.9609&link_type=DOI) 

35. 35.Zahabi M, Kaber DB, Swangnetr M. Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation. Hum Factors 2015;57(5):805–34. doi: 10.1177/0018720815576827
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1177/0018720815576827&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=25850118&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

36. 36.Butler JM, Gibson B, Lewis L, et al. Patient-centered care and the electronic health record: exploring functionality and gaps. JAMIA Open 2020;3(3):360–68. doi: 10.1093/jamiaopen/ooaa044
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1093/jamiaopen/ooaa044&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=33215071&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

37. 37.Zheng K, Ratwani RM, Adler-Milstein J. Studying Workflow and Workarounds in Electronic Health Record-Supported Work to Improve Health System Performance. Ann Intern Med 2020;172(11 Suppl):S116-S22. doi: 10.7326/M19-0871
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.7326/M19-0871&link_type=DOI) 

38. 38.Bloom BM, Pott J, Thomas S, et al. Usability of electronic health record systems in UK EDs. Emerg Med J 2021;38(6):410–15. doi: 10.1136/emermed-2020-210401
    
    [Abstract/FREE Full Text](http://medrxiv.org/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIzOC82LzQxMCI7czo0OiJhdG9tIjtzOjUwOiIvbWVkcnhpdi9lYXJseS8yMDI0LzEyLzAxLzIwMjQuMTEuMjcuMjQzMTgxMDkuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

39. 39.Scott K, Hathaway E, Sharp K, et al. The Development and Evaluation of an Electronic Health Record Efficiency Workshop for Providers. Appl Clin Inform 2020;11(2):336–41. doi: 10.1055/s-0040-1709509
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1055/s-0040-1709509&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=32375195&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

40. 40.Longhurst CA, Davis T, Maneker A, et al. Local Investment in Training Drives Electronic Health Record User Satisfaction. Appl Clin Inform 2019;10(2):331–35. doi: 10.1055/s-0039-1688753
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1055/s-0039-1688753&link_type=DOI) 
    
    [PubMed](http://medrxiv.org/lookup/external-ref?access_num=31091545&link_type=MED&atom=%2Fmedrxiv%2Fearly%2F2024%2F12%2F01%2F2024.11.27.24318109.atom) 

41. 41.Tierney AA, Gayre G, Hoberman B, et al. Ambient Artificial Intelligence Scribes to Alleviate the Burden of Clinical Documentation. NEJM Catalyst 2024;5(3):CAT.23.0404. doi: 10.1056/cat.23.0404
    
    

42. 42.Nahar JK, Kachnowski S. Current and Potential Applications of Ambient Artificial Intelligence. Mayo Clinic Proceedings: Digital Health 2023;1(3):241–46. doi: 10.1016/j.mcpdig.2023.05.003
    
    [CrossRef](http://medrxiv.org/lookup/external-ref?access_num=10.1016/j.mcpdig.2023.05.003&link_type=DOI)