Abstract
Introduction Medication errors during paediatric resuscitation are thought to be common. However, there is little evidence about the individual process steps that contribute to such medication errors in this context.
Objectives To describe the incidence, nature and severity of medication errors in simulated paediatric resuscitations, and to employ human reliability analysis to understand the contributory role of individual process step discrepancies to these errors.
Methods We conducted a prospective observational study of simulated resuscitations subject to video micro-analysis, identification of medication errors, severity assessment and human reliability analysis in a large English teaching hospital. Fifteen resuscitation teams of two doctors and two nurses each conducted one of two simulated paediatric resuscitation scenarios.
Results At least one medication error was observed in every simulated case, and a large magnitude or clinically significant error in 11 of 15 cases. Medication errors were observed in 29% of 180 simulated medication administrations, 40% of which considered to be moderate or severe. These errors were the result of 884 observed discrepancies at a number of steps in the drug ordering, preparation and administration stages of medication use, 8% of which made a major contribution to a resultant medication error. Most errors were introduced by discrepancies during drug preparation and administration.
Conclusions Medication errors were common with a considerable proportion likely to result in patient harm. There is an urgent need to optimise existing systems and to commission research into new approaches to increase the reliability of human interactions during administration of medication in the paediatric emergency setting.
Strengths and limitations of this study
This study is one of the first to use HRA methods to link task discrepancies with resultant medication errors, as well as to link these discrepancies directly to potential harm. This effort has demonstrated that a significant fraction of the burden of error in the paediatric emergency drug administration process originates during the preparation and administration phase and that most of these errors are likely to be undetected in clinical practice.
This study was subject to a number of limitations. Although we went to considerable lengths to replicate the paediatric emergency environment, the simulation environment cannot truly reflect the clinical environment during a genuine emergency.
Furthermore, this study was conducted at a single site and participants were not blinded to the purpose of the study, so it is potentially subject to preparation bias.
Participants were recruited from the paediatric emergency unit, intensive care unit and general paediatrics ward and had variable experience of emergency cases. However, all participants worked in clinical units that manage critically ill children.
- Paediatrics
- Paediatric A&E and ambulatory care
- Paediatric anaesthesia
Introduction
Background
Medication errors are among the leading cause of avoidable harm in healthcare worldwide 1 and up to three times more common in children than in adults.2 The paediatric emergency environment, characterised by urgency and fraught with interruptions, is one of the clinical areas most vulnerable to error. Medication administration in emergencies is complex as it requires successful interactions between different teams of medical and nursing staff, as well as between individual members of these teams. An additional challenge relates to obtaining relevant medication information and translating this into the required dose and concentration of the correct drug to be administered by the correct route for the clinical indication, all in a necessarily short space of time. Medication errors in general, and medication administration errors in particular, are both under-detected and under-reported, such that little is known of their incidence or impact during resuscitation. However, medication errors have been reported in 7 out of 10 simulated paediatric resuscitations,3 with other recent simulation studies suggesting 26% 4 to 70% 5 of administered medicines being given at the wrong dose. Laboratory studies analysing syringes prepared for anaesthestic use have found at least 15% to be greater than 20% discrepant from the intended drug concentration.6
The broader, systems view of medical error, heralded by the Institute of Medicine’s “To Err is Human” report, saw the widespread adoption of Reason’s organisational accident model 7 in healthcare. More recently, human reliability analysis (HRA) techniques, previously commonplace only in other high risk industries, have become increasingly used in healthcare.8,9 HRA is based on the understanding that neither humans nor systems can be error-proof, and asserts that to improve safety and reliability, a thorough analysis of system vulnerabilities at a task level is needed, taking into account human-human and human-machine interactions.10,11 Medication safety researchers have previously used a HRA technique, the systematic human error reduction and prediction approach (SHERPA), to identify system vulnerabilities in ward-based medication administration,12 anaesthesia13 and general surgery.14 However, this approach has not been used quantitatively in medication safety research and has not been applied to paediatric resuscitation.
Our objectives were to describe the incidence, nature and severity of medication errors in simulated paediatric resuscitations, and then use HRA to understand the contributory role of individual process step discrepancies with a focus on those contributing to large magnitude and/or clinically significant errors.
Methods
Study design and setting
This prospective observational study was conducted April to November 2017 in a medical simulation facility within a large teaching hospital. The hospital has a paediatric emergency department (seeing 27,000 children each year) and a comprehensive paediatric inpatient service (admitting 5,000 each year). The hospital used electronic prescribing in the inpatient setting, but during resuscitations, medications were more commonly ordered on paper prescription charts. We recruited resuscitation teams of four clinicians, that were randomised to participate in one of two standardised simulated paediatric resuscitation scenarios. The study was approved by the Health Research Authority and the hospital concerned. NHS ethics approval was not required. Participants gave written informed consent.
Patient and Public Involvement
The research team held a workshop with parents to get their feedback on the proposal, develop the PPIE plans, and identify future important areas for research. We actively sought attendees through INVOLVE’s “People in Research” website, social media and Imperial College London’s existing networks. Our team has also participated in a PPIE event run by the Royal College of Paediatrics and Child Health, in collaboration with MedsIQ (http://www.medsiq.org) and Medicines for Children (http://www.medicinesforchildren.org.uk), two UK-based paediatric medication safety initiatives.
Participants
Eligible participants were a convenience sample of medical and nursing staff from the departments of paediatrics and paediatric emergency medicine at the study hospital. Participants were assigned into teams comprising a senior doctor (a specialist registrar, with at least a year of prior experience as a registrar), a junior doctor, a senior nurse (with at least 5 years’ nursing experience) and a junior nurse.
Clinical scenarios
The two scenarios were:
Prolonged status epilepticus in an 8-month-old, 8kg child
Presumed meningococcal sepsis in a 10-month-old, 9kg child
The two scenarios (Appendix 1) were designed by a collaboration of paediatric nurses, emergency physicians, intensivists, general paediatricians, and anaesthetists. Face validity was established by an independent expert panel of six, with representation from each of these professional groups, including two lead paediatric clinical nurse educators. The two scenarios were determined to be similarly demanding.
A simulated paediatric resuscitation bay was created. The mannequin used was a SimBaby Version 2 (Laerdal Medical, Stavanger, Norway), and the syringe pump stack consisted of Alaris PK MK4 units (Becton Dickinson, Franklin Lakes, USA). All relevant print materials (e.g. formularies and protocols) and hospital information technology systems were available. Participants were requested to prescribe, prepare and administer medications exactly as per usual practice, to use mobile applications or websites as they would in clinical practice, and to telephone specialist colleagues if required. A paediatric intensivist, the hospital lead for paediatric simulation, ran the scenarios. She provided standardised clinical information as live feedback and answered questions regarding the child’s response to treatment or their current condition when needed.
Data sources and measurement
A Scotia Medical Observation System (Smots, Scotia UK, Edinburgh, UK), with two 3-axis, ceiling-mounted video cameras, and three mobile, high-definition cameras equipped with boom microphones, was used. Both nurses in each team wore head-mounted high-definition video cameras (GoPro Inc, California, USA). The video recordings were analysed by a research nurse with ten years’ experience in paediatric intensive care.
Outcome measures
We used the term ‘medication error’ to describe an overall error with respect to a particular drug administration as a whole, and the term ‘discrepancy’ to refer to observed deviations at the level of the individual task. A discrepancy may or may not lead to a medication error.
Variation in clinician practice and human adaption to the complex process of medication ordering, preparation and administration typically results in many minor task discrepancies that may not individually, or even in combination, result in a medication error or patient harm.15 To identify the most important task discrepancies, we assessed all observed discrepancies to establish the extent to which they may or may not have contributed to any resultant medication errors.
Figure 1 summarises the study objectives and associated analyses.
*SHERPA = systematic human error reduction and prediction approach
Medication errors
Medication errors included any errors in dose, administration rate, concentration, drug, route or method of administration, timing, or delay in administration. Operational definitions for each of these are given in Appendix 2. Briefly, dosing errors were defined as a greater than 10% deviation from the recommended dosing range (DRDR) 16 at the study site. Any deviation from the recommended rate of administration (DRDRate) was calculated in a similar manner and deviations of more than 10% were considered to be medication errors. Deviations from the recommended concentration (DRC) of greater than 10% from the concentration specified in local guidance were also included as medication errors.
To identify delayed administrations, the time taken for the dose to be ‘ready for delivery’ (tDRD) was calculated as the time for the doctors to obtain any medication information required plus the nurse-led preparation time. The time to be ready for deliver was considered ‘prolonged’ when a particular team took more than double the median time for that specific drug across the entire study without clinical cause for the delay as determined by the nurse assessor. For example, if a medication administration was interrupted to reassess the patient clinically or to administer another medication as a priority, a prolonged tDRD would be excluded as an error on clinical grounds.
Severity assessment
There are few validated tools that can be used to assess the severity of medication errors without knowledge of patient outcomes and that are thus usable in simulated studies 17. One of these tools is that of Dean and Barber, based on 4-5 experts independently assessing each error on a 0 to 10 scale, and their mean score used as an index of severity. Mean scores under 3 suggest errors of minor severity, those between 3 and 7 as moderate and those greater than 7 as severe.18 We therefore used this approach, with two paediatric intensivists, one paediatric anaesthetist, one senior critical care nurse, and one senior clinical pharmacist assessing each error.
Discrepancies at the level of the task
A hierarchical task analysis (HTA) was developed based on a similar framework for ward-based medication administration12 and assessed for face validity by five senior nurses in the study hospital. A generic human error taxonomy, based on SHERPA external error modes19 with one additional error mode, ‘information not sought’, was used to code observed discrepancies against the HTA.
Where there were more than two discrepancies at a single step for a specific administration, the nurse assessor made a subjective assessment of which had the greater overall consequence, and assigned an error mode (Figure 2) to that discrepancy only. To capture ‘root-cause’ system vulnerabilities, steps where an action was performed correctly, but which perpetuated a previous medication error, were not classed as discrepancies.
Significance assessment of task discrepancies
All task discrepancies were classified by the nurse assessor according to the contribution made by the discrepancy as follows:
No contribution: the discrepancy did not contribute to a medication error
Minor contribution: some contribution made to a medication error
Major contribution: the task discrepancy led directly to a medication error
Data management and analysis
Medication errors were described according to the type of error, method of administration and stage of the medication use process in which the error occurred. Error rates were calculated using the total number of applicable administrations as the denominator.
Step discrepancies were presented as counts grouped by task and contribution to medication error. Discrepancy rates were calculated as the percentage of discrepancies that made a major, minor or no contribution to an error, with the number of observed discrepancies at each process step as the denominator. Of those discrepancies making major contributions to medication errors, the proportion that led to clinically significant errors (severity score > 3) and/or large magnitude errors (DRDR or DRDRate > 25%) was also calculated.
There is no literature that quantifies the extent to which a step discrepancy having a minor or major contribution to an error is of greater significance than a step discrepancy that makes no contribution to an error. For a weighted, ‘heat map’ HRA analysis, it was therefore necessary to attribute different weights to discrepancies that resulted in error to those that did not. Sub-step discrepancies were therefore weighted, agreed by expert panel, as follows:
No contribution: weight = 1
Minor contribution: weight = 10
Major contribution: weight = 40
The total weighted significance score for each step was thereby calculated for each error mode.
Inter-observer reliability
One of the 15 simulations was re-analysed by an additional independent nurse. Spearman’s rank correlation coefficient was calculated for continuous variables and Cohen’s Kappa for categorical variables.
Results
Data were collected during 15 simulations according to participant availability, eight for prolonged seizures, and seven for meningococcal sepsis. Participants comprised 30 doctors and 30 nurses (Table 1), each of whom completed one simulation. Inter-observer reliability analysis revealed perfect agreement for categorical variables, and high agreement for continuous variables with a Spearman’s rank correlation coefficient 0.957 or above.
Medication errors
Participants conducted 180 medication administrations. Overall, errors were observed reaching the patient for 52 drug administrations (29%) and at least once in every simulation. Of these errors, 30 (58%) were assessed as being of minor severity, 16 (31%) as moderate and 6 (12%) as severe. There were 27 large magnitude errors (52% of all errors), in which the DRDR / DRDRate was greater than 25%. Of all erroneous administrations that reached the patient, only two (4%) were noticed by staff after administration and therefore may have been reported in clinical practice. A detailed error analysis is provided in Table 2 and description of the ten most severe errors in Table 3.
Hierarchical task analysis
The full HTA is shown as Figure 3 and shows all steps assessed in the paediatric emergency drug administration process.
Errors and discrepancies by stage of medication use and process sub-step
Overall, 884 step discrepancies were observed, excluding dependent downstream discrepancies after an initial discrepancy. Of these 884 step discrepancies, 174 (20%) were linked to a medication error, with 70 (8%) assessed as making a major contribution to an error, 104 (12%) making a minor contribution, and 710 (80%) making no contribution.
Figure 2 shows the significance-weighted HRA data represented as a heat-map demonstrating the relative contributions of discrepancies at each step and by each error mode to medication errors. Table 4 summarises the discrepancy counts per step as well as the percentage of both large magnitude and clinically significant errors with a major contribution made at each HTA step.
Errors and discrepancies during medication ordering
We observed 170 discrepancies during the ordering phase. Five of the 22 clinically significant medication errors were due to discrepancies during medication ordering, with three of these due to incorrect dose information retrieval from the British National Formulary for Children. The majority of the remaining discrepancies (136) were due to incomplete verbal medication orders based on which drug preparation commenced, two of which resulted in medication errors. Of the 180 written medication orders examined, there were six discrepancies observed, all of which were corrected by the clinicians and therefore made no contribution to any dosing errors, but did result in one delayed administration.
Errors and discrepancies during medication preparation
During medication preparation, 310 discrepancies were observed, representing 35% of all observed discrepancies. These contributed to one medication error involving the wrong drug, ten dose errors and 20 diluent or dilution errors. The retrieval of preparation and administration information from the online IV medications guide was the step mostly likely to contribute to medication errors during medication preparation, with 42 discrepancies (19 major contribution to a medication error, one minor), resulting in nine clinically significant medication errors (severity score > 3). The retrieval of incorrect information and taking an excessively long time to identify the correct information within the guidance were the most common discrepancy error modes.
There were seven discrepancies when converting milligrams to millilitres of undiluted drug, and 18 discrepancies (six making a major contribution) when, after having made the correct calculations, nurses withdrew either the incorrect amount of undiluted drug or the incorrect amount of diluent into the syringe.
Double-checking
Overall, 259 discrepancies were observed during the double-checking phase, 72 (28%) of which made a minor contribution to a medication error. Checking the route and method (e.g. infusion or bolus) of administration was the most frequently omitted.
We observed 29 medication errors that were made during medication ordering and preparation but had not been yet administered to the patient at the point of double checking. These errors were thus potentially ‘interceptable’ but all ultimately reached the patient. Of these errors, in 14 cases, the double-checking interaction between the nurses included the incorrect step, but failed to identify it as incorrect.
Errors and discrepancies during the administration phase
Of all observed discrepancies, only 28 (3%) occurred during administration. These resulted in 11 wrong rate errors (five moderate, two severe), nine wrong method errors (three moderate, one severe) and one severe wrong time error. Discrepancies during the administration phase constituted a third of all discrepancies that made a major contribution to a clinically significant error.
Infusions in particular were prone to administration errors. Of the 17 discrepancies observed during infusion rate calculations or when programming the infusion pump for intermittent infusions, 14 were of major consequence, and accounted for 23% of all clinically severe errors. Seven discrepancies (four making a major contribution to a medication error) occurred when determining the delivery rate for continuous infusions.
Discussion
This prospective observational study is the first in paediatric emergency medicine to include a quantitative HRA, allowing identification of the task discrepancies with the greatest contribution to medication error. We identified at least one medication error in all 15 simulations, and a large magnitude or clinically significant error in 12 of these.
Comparison with previous literature
Historical heterogeneity of the definitions of medication error and the variability in reporting metrics make comparison with previous literature difficult.20,21 Additionally, there are few simulated studies and no relevant clinical studies in paediatric resuscitation, making comparator data scarce.
Prescribing error rates in the emergency setting have been reported to be 10.1% and 16% 22,23 of all orders; our study reports a lower rate of 5%, although this difference may be at least partly due to different error definitions. Our study instead suggests that preparation and administration errors may be more common. Other simulated studies have reported error rates for the administration of intravenous bolus medication of between 15.5% and 26.5% 3,4,24; in our study, it was 31%. The referenced studies, however, reported only on dose errors, and not any other medication error types. Only seven out of the 24 medication errors we observed for bolus doses were dose errors. Medications given by intermittent infusion were the most error-prone in our study. There are no studies that investigate emergency administration of intermittent infusions in sufficient detail to provide a basis against which to compare this finding.
Medications given by continuous infusion are potentially the most complicated in paediatric emergencies. In addition to the preparation steps for intermittent infusions, staff generally have to convert infusion rates in milligrams or micrograms per kilogram per minute to infusion rates of millilitres per hour. A recent trial of a digital application reported errors in 70% of continuous vasopressor infusions 5 in the control arm. However, despite the increased cognitive demand, we observed the lowest incidence of medication error for continuous infusions, at 18%. Administration of continuous infusions in our hospital seemed to be relatively well supported by an online / paper tool 25.
Implications for research and practice
This study highlights the need for research to optimise clinicians’ use of electronic resources containing medication preparation and administration information. We were not able to pinpoint the precise steps at which the current electronic intravenous medications guidance system in the study hospital proved vulnerable to misinterpretation. Research to further understand the steps that need attention may serve as a useful basis from which to refine, and if needed, redesign such systems.
This study reaffirms that performing complex arithmetic in high-stress clinical environments is a considerable contributor to medication error.26 With the purpose of addressing medication safety in paediatric resuscitation, the literature has been dominated by studies looking at ‘resuscitation aids’, most commonly length-based tapes.16,27,28 These aids couple weight estimation with a suggested dose for a limited number of medications, but without providing preparation and administration support. It is not likely that length-based tapes would have decreased the rate of medication error in this study. Further clinical research is required to determine the effectiveness of new digital tools that do support preparation and administration, such as those that have shown promising results in simulated studies.5,29
Human factors methods have been used in other high-risk industries to define system vulnerabilities for building safer systems. By using quantitative HRA, this study provides evidence for the prioritisation of research efforts directed towards new interventions to address the most important system weaknesses.
In terms of implications for practice, one of the most unexpected findings in this investigation was the uncovering of ‘purely mechanical’ task discrepancies resulting in medication errors. During drug preparation, clinicians were observed drawing up incorrect volumes of medications or diluents even though all calculations were correct. This suggests that efforts seeking to address medication safety in cognitively demanding environments using clinical education strategies or contemporary technologies must do so without disregarding the seemingly ‘simplest’ aspects of drug preparation. The reliability of information exchanges between healthcare professionals similarly needs improvement. Verbal medication orders in particular are inconsistent and error-ridden. Particular attention should be paid to medication orders given verbally in the emergency setting, using approaches such as the recipient verbally confirming the medication and dose being prepared. More importantly however, there is an urgent need for research to explore how to bring greater effectiveness to checking and double-checking more broadly. These are steps intended to defend patients from error, but which are too often ineffective.
Conclusions
Overall, we identified errors in 29% of all simulated medication administrations, only two of which were detected by participants, with 40% of these likely to result in moderate or severe harm. HRA revealed a number of error-prone steps, many of which occurred during preparation and administration of correctly ordered medications. The task most likely to result in erroneous medication administration was ineffective retrieval of correct medication preparation and administration instructions from intravenous medication guidance.
This study has highlighted an urgent need to optimise existing systems and to commission new approaches to increase the reliability of human interactions with the emergency medication administration process.
Data Availability
All data relevant to the study are included in the article or uploaded as supplementary information
Ethical approval
The Health Research Authority approved this study.
Conflicts of interest
The Helix Centre at Imperial College London is leading an effort in collaboration with the British National Formulary developing digital tools in an attempt to improve paediatric medication safety. NA has written two patents describing syringe labelling techniques in medication safety. BDF supervises a PhD student part funded by a supplier of hospital electronic health record systems, and has received funding from Pfizer for delivering teaching at a one-off symposium on medication safety.
We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including the timing of publication, with respect to intellectual property. In so doing we confirm that we have followed the regulations of our institution concerning intellectual property.
Acknowledgements
We would like to thank all clinical staff who participated in the simulations. It is our hope that the enthusiasm and extraordinary open-mindedness our dedicated and deeply competent colleagues demonstrated in making this study possible will lead to safer systems and innovations in paediatric medication safety.
This study was funded by the NIHR-Imperial Patient Safety Translational Research Centre, with infrastructure support from the NIHR Imperial Biomedical Research Centre. Support was also received from the Resuscitation Council (UK).
References
