Has the implementation of time-based-targets for emergency department length-of-stay influenced the quality of care for patients? A systematic review of qualitative literature

inclusion criteria

Original qualitative or mixed methods research studies were eligible for inclusion if they described a state, regional or national time-based-target for emergency length-of-stay; were published between 2000 and 2019 and reported knowledge, attitudes or experiences regarding the target. Included papers, in any language, needed to describe emergency department patients and the perspectives of stakeholders who interacted with emergency medicine before and after the introduction of the targets. Data could be acquired through interviews or focus groups. Review articles, surveys, simulation studies, secondary reports of other studies, news articles, letters and editorials were excluded.

search strategy

Structured searches were undertaken in the Cochrane library, Medline, Embase, Cumulative Index to Nursing, and Allied Health Literature, Emerald, Abstracted Business Information/Inform and Informit (final search on October 23rd 2019). A search strategy was developed using text words and index terms related to synonyms for time-based targets, emergency departments, and any term related to outcomes, quality or evaluation. Search strategies are in Appendix 1. Authors were consulted regarding unpublished work and hand searches were performed.

study screening

Abstracts were placed in a bibliographic management tool (Mendeley)⁴⁰ and duplicates were removed. Remaining abstracts were uploaded to Covidence systematic review software⁴¹. Full texts screening followed abstract screening for all abstracts potentially eligible for inclusion. Two reviewers screened each abstract and full text article, (blinded to the second reviewer assessment). Inter-rater agreement during initial full-text screening was measured using a kappa statistic. Disagreements were resolved either by a consensus discussion between reviewers or a third reviewer. Reasons for excluding publications were recorded during full-text screening.

data extraction

The following data were extracted: title; journal; date of publication; sponsorship source; country; setting; lead author name, institution, and contact details; methodology (design, when data was collected, analysis and synthesis); participant inclusion and exclusion criteria; recruitment method; source of data, baseline characteristics of participants and sites; type of target; details of the target; date of target introduction; incentives to meet targets and outcomes from the study. Illustrative quotes from respondents were extracted from each included study. Each screening reviewer verified the extracted data.

quality appraisal of individual studies

Individual study demographic and methods data were collected using a data extraction form in Covidence. Data regarding study outcomes, quality and risk of bias were collected on a data extraction form, based on the Critical Appraisal Skills Program (CASP)⁴² requirements for qualitative study evaluation. Domains assessed were: aims, methodology and design, recruitment, data collection, analysis, outcomes, and an assessment of overall risk of bias. The template is available in Appendix 2. Two reviewers independently extracted data and one reviewer combined the results. Disagreements were resolved through discussion or by a third reviewer. No studies were excluded from the review based on this assessment.

data synthesis

Two emergency physician reviewers individually identified all the key outcomes for each study, and interpreted each outcome as positive, negative or neutral, with regards to the impact of the targets on quality of care. Reviewers weren’t provided with definitions of positive/negative/neutral. A thematic synthesis approach was used to analyse and synthesise individual study outcomes. Reviewers iteratively classified all the findings from the studies into themes. Regular meetings were held throughout the data synthesis process for reviewers to compare and agree on descriptive/emerging themes and interpretations. Quotations were selected to illustrate each theme.

assessing confidence in the review findings

Assessment of confidence in the finding was undertaken using the Confidence in Evidence from Reviews of Qualitative Research (CERQual)⁴³, which applies the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach to qualitative studies^44,45. CERQual assesses each review finding against four domains: methodological limitations⁴⁶, relevance⁴⁷, coherence⁴⁸ and adequacy of data⁴⁹. Reviewers categorised confidence in each finding as high, moderate, low or very low, based on the assessments for each CERQual component⁵⁰, with equal weight given to each component. High confidence in a synthesised finding indicates that it is a reasonable representation of time-based targets. If there are concerns with regard to any of the above four domains, then confidence in the finding is downgraded. Where the domains overlap, reviewers avoided downgrading a review finding twice for the same concern across domains. Two reviewers individually undertook the assessment of confidence in findings using the CERQual approach.

study selection

The flow of studies through the review is described in Figure 1. Initial inter-observer full text inclusion agreement was 98/121 (81%), κ=0.60 (95% CI 0.45-0.74), indicating moderate agreement.

• Download figure
• Open in new tab

Fig 1. PRISMA Flow Diagram, ED time-based-targets, flow through qualitative Systematic review

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097

For more information, visit www.prisma-statement.org.

study characteristics

Thirteen studies were included in the review, and their characteristics including the setting, perspectives of respondents and the target intervention are presented in Table 2.

Six studies were from Australia^51-56, three from New Zealand^57-59, three from England^60-62, and one from Canada⁶³. The studies were qualitative^{52,53,56,59-63} or mixed methods^{51,54,55,57,64}. The included study settings were all single-payer, government-run health care systems, in English speaking, high-income countries. All studies purposefully recruited participants, representing a wide range of stakeholders. In total 617 interviews were reported. Three studies derived from the same participant interviews in New Zealand^57-59; four derived from the same participant interviews in Australia^51-54.

Most participants were emergency and hospital staff, including managers, senior and junior medical and nursing staff, allied health professionals, clerks, and project team members. One study⁵⁵interviewed representatives from medical bodies, and members of the public.

The included studies were generally assessed as being of high quality with low risk of bias. A few studies had a higher risk of bias due to small study populations. The appraisal and risk of bias for each included study is described in Table 3.

We identified 53 outcomes from the included studies and categorised each into a theme. Themes explored perceptions of the impact of time-based-targets on quality and safety of care, timeliness of investigations or treatment, length-of-stay, receiving care, representations, unintended consequences, patient and staff experiences, and enablers and barriers to implementing time-based-targets. For each theme, we have described key findings and our confidence in each finding. A narrative review of each theme is presented below. A detailed analysis is shown in Table 4.

quality and safety of ED care

Overall 12 studies contributed to this theme^51-56,58-63. We have high confidence in this finding due to a large number of participants, representing a wide range of settings and perspectives. Implementation of time-based-targets for emergency length-of-stay had both positive and negative impacts on the capacity to deliver emergency care.

Respondents reported positive experiences when targets were used as a lever to improve patient flow and timely care. Clinicians and managers who perceived that the introduction of targets had a positive impact on quality and safety of care noted that pursuit of the time-based-targets led to improved processes. These allowed emergency nurses and physicians to work more efficiently, refer earlier and perform fewer tests, leading to expedited delivery of patient care ^55,61,62. In these facilities, sources of delay were investigated and hospitals invested in quality improvement activities to meet the targets, such as new infrastructure and new staffing roles^54,62,63. One Australian emergency physician observed:

“It probably has a positive [impact] in that we’re making decisions and getting patients to the ward earlier…”⁵¹

In England, an emergency clinician noted:

“Before the target, people were stacked in the corridors on trolleys. Care is better because we have got rid of the corridors of shame.”⁶⁰

Negative experiences occurred when targets were prioritised over patient needs. Health staff in all roles reported this issue^53,55,56,62. Hasty decision-making impacted negatively on patient assessments, increased test ordering and increased misguided care by junior doctors⁵³. Allied health staff reported that medical patient evaluations were less detailed⁵⁹. Clerical staff administration errors also increased; which was attributed to productivity pressures⁵⁹.

“…somebody’s in pain and you need to get them on the ward within those 4 hours - do you move them or not?” Emergency nurse, England⁶¹

There was a perception of increased patient churn and a chaotic environment in wards, which made it more difficult for clinicians to maintain quality of care, exposing patients to increased risk⁵⁴. Ward staff felt they were over-census at times, reducing patient privacy and safety⁵⁹.

access block and overcrowding

Nine studies contributed to this theme^{51-56,58,60,61}. We have low confidence in this finding as quantitative methodology is better suited to exploring this theme, leading to moderate methodological concerns about this finding.

Many emergency staff and managers found that patient flow and access block improved with the targets^51,53,55,61even if they did not meet the target. This was attributed to process changes driven by the creation of a closely monitored target^56,59. Different hospitals used different strategies to meet targets. Some sites implemented rapid assessment models, streaming to short-stay units, use of new coordinator roles and improved discharge processes. These changes had varying success in improving access block and ED overcrowding. There is insufficient data from these studies to comment on which models of care are most successful in reducing access block and overcrowding. Targets didn’t improve access block for psychiatric patients in Australia⁵⁵. New models of care were found in hospitals in all settings and geographical locations. They were implemented at a local level and there was significant variation in the approach to meeting targets.

Emergency staff were concerned that the policy could worsen emergency overcrowding through increased emergency attendances (because consumers thought there would be shorter queues as a result of the policy)⁶¹, increased representations (having discharged patients too early)⁵⁹ and higher admission rates leading to reduced inpatient bed capacity^55,56. These concerns were disproven in a quantitative systematic review on the topic (Under review, BMJ Open).

patient experience

Six studies contributed to this finding^{52,55,56,59,61,63}. We have very low confidence in this finding due to inadequate data representing the patient voice. Only one study includes a consumer perspective.

In Western Australia, feedback from the Health Consumers’ Council was very positive⁵⁵. Clinicians in New Zealand, England, and Canada reported fewer patient complaints^59,63 and felt that patient satisfaction increased⁶¹. Some clinicians were concerned that the targets reduced the capability of emergency staff to build rapport with patients⁵².

staff morale and workload

Eight studies contributed to this finding^{51-55,57,59,62}. We have high confidence in this finding because the contributing data represents a wide range of settings and perspectives. While some of the included studies had methodological limitations, there was agreement between findings from these studies compared to others that were assessed as being of high methodological quality.

Clinicians reported a range of positive and negative experiences associated with the introduction of targets. Many doctors and nurses experienced increased stress and lowered morale due to increased pressures associated with meeting the target. For example, one Australian emergency director felt that:

“all you’re saying to the staff is, ‘You’re not good enough. You don’t work hard enough. You can’t do your job fast enough’”⁵¹

They also described a sense that they were being micro-managed and under surveillance^{51,53,59,61,62}. More than one respondent describing feeling like they were being watched by ‘Big Brother’^54,61. Many participants reported an increase in their workload in relation to the targets, including increased documentation requirements^52,53 and reduced efficiency due to ‘workarounds’⁶². This had the effect of reducing nursing morale and increasing staff turnover, with one nurse reporting that junior nurses were leaving:

“because they feel they cannot cope with high pressure work all the time”⁶¹

Furthermore, patient awareness of the target could increase stress for emergency nurses because patients felt enabled to challenge delays⁶¹. Junior doctors reported feeling bullied by nurses allocated to the role of time-based-target compliance^51,52,55,62. Ward staff felt increased pressure to discharge patients from hospital wards⁵⁹.

Conversely, some emergency staff reported less stress, more control over their environment and an enhanced ability to manage increased attendances^52,53. This was attributed to improvements in staffing and resources, increased control over the admission process and improved autonomy for emergency medicine regarding decision-making^60,62. Administration staff morale and stress didn’t change in Australia^51-54.

intrahospital and interdepartmental relationships

Nine studies contributed to this finding^{51-56,58,59,62}. We have high confidence in this finding because the contributing data represents a wide range of settings and perspectives. While some of the included studies had methodological limitations, both higher and lower quality studies presented similar findings.

Time-based-targets impacted on relationships and the interface between emergency medicine, other hospital departments and hospital leadership. At many sites, the emergency department was deemed responsible and accountable for delivering targets, despite a lack of hospital system and process change within the hospital, which exacerbated intra-hospital tensions^52,59,60.

clinical education and training

Six studies contributed to this finding^{51,53-55,60,61}. We have very low confidence in this finding due to inadequate data from junior doctors.

Emergency clinician and managers felt that medical education training environments had deteriorated, with fewer training opportunities and less time for junior doctors and nurses to learn^53-55,60,61;. Hospital managers deprioritised training and research, compared to targets⁵³. One Australian emergency physician observed:

“I think the biggest impact that it’s had is that it’s actually sunk all our resources into trying to achieve that one target so that other aspects of our normal work which we think are important, such as education and research, are just continually being eroded”⁵¹

Some participants reported that role restructuring in order to meet the targets led to greater emergency team integration, having a positive impact on medical education, and improved educational opportunities⁵³. Another Australian emergency physician reported:

“There is very strong evidence that actually working with the junior doctors is actually better teaching than letting them bumble through it”⁵⁵

In the UK, a senior clinician described both the positive and negative impacts of targets on training and education:

“In general, the target has had a deleterious effect [on training] not as much opportunity to stand and talk… It takes them twice as long to do procedures—so we are not always able to let them do these. However, what was training like when we had 80 patients in the department?”⁶⁰

gaming

Five studies contributed to this finding^53,54,57-59. We have moderate confidence in this finding as the data comes from New Zealand and Australian settings only.

Gaming has been described as “making performance appear better when it is not”^57,66 Clinicians and managers found that gaming occurred in order to meet targets^53,54,58,59. This included avoiding target breaches by diverting patients to short-stay units when ward beds weren’t available⁵⁸. This behaviour is described by one Western Australian emergency nurse as:

“The patient is about to breach, just get them into a bed, any bed, instead of taking a little bit longer to make an accurate disposition decision”⁵⁵.

Gaming was universal and occurred more frequently when there was pressure from senior hospital management staff on ED or the “whole-of-hospital” to meet the target, but at lower levels when ED staff did not feel pressure to meet the target⁵⁷.

enablers and barriers to achieving targets

Thirteen studies contributed to this finding51-63. We have high confidence in this finding because the data reflected a wide range of settings and perspectives, with a high level of consistency in outcomes.

There was a high degree of consensus that successfully achieving the targets required a top-down and whole-of-hospital approach^{51,58,60,63,67}. Describing the targets as ‘emergency time-based-targets’ was felt to be misguided, as efforts to improve timeliness were impacted by processes that involve the whole hospital system^55,56,62.

Factors that were thought to be required for successful change management included careful pre-planning, staff education and changes tailored to the local environment⁵⁴, provision of short-term project support officers55 63, learning from other sites59 and executive and managerial buy-in⁵⁸.

Hospitals introduced a broad range of new processes and models of care. These included changes to the handover process between emergency and inpatient units⁵⁴ and new targets for patient work-up and ward bed allocation58. Improvements in out-of-hours discharge planning and patient discharge lounges were helpful58. Extended opening hours support services (e.g. diagnostics and pharmacy) was also required61

Barriers to target achievement included lack of ED authority regarding admissions and insufficient support from logistics or other areas of the hospital⁵⁴. Pursuit of diversion strategies was largely ineffective⁵⁸. A need to engage community referrers in future acute care service planning was reported61. Financial resources and physical changes in capacity and layout were required to implement the changes. Without this, sites had inadequate resources to implement the policy or manage the workload^53-55,62.

Hospitals in New Zealand rostered more doctors, nurses and orderlies in both emergency departments and wards⁵⁸. In England, emergency nurse practitioner numbers increased to see “minor” patients more efficiently262. The skill mix of emergency clinicians and inpatient ward staff was identified as important, with a need for more senior and experienced clinicians to be involved earlier in the course of the patient journey^54,55,60. Hospitals upgraded communications and information technology systems to meet time-based-targets, at times improving flow management, but at other times reducing efficiency^54,58,62.

overall

The variation in experience was not dependent on jurisdiction, country or time target or threshold with respondents from hospitals across Australia, NZ, Canada and the UK all contributing both positive and negative impressions, experiences and impacts on care quality. These experiences were universal across jurisdictions. The impact of targets was associated with how the various targets were implemented and achieved. When applying targets, positive effects on quality of care were noticed when health services adopted a whole-of-hospital response and ensured that meeting the target did not occur at the expense of patient care activities, or training for junior clinicians. Negative effects were experienced by staff when the target was prioritised over patient care, there wasn’t whole hospital engagement and when resources weren’t adequate to implement and maintain target achievements (such as physical layout and capacity, increased staffing, processes, change management personnel, information technology support, inpatient beds, and inpatient engagement).