Abstract
Background Multiplex molecular diagnostic panels have greatly enhanced detection of gastrointestinal pathogens. However, data on the impact of these tests on clinical and patient-centered outcomes are limited.
Methods We conducted a prospective, multicenter, stepped-wedge trial to determine the impact of multiplex molecular testing at five academic children’s hospitals in children presenting to the ED with acute gastroenteritis. Caregivers were interviewed on enrollment and again 7-10 days after enrollment to determine symptoms, risk factors, subsequent medical visits, and impact on family members. During the pre-intervention period, diagnostic testing was performed at the discretion of clinicians. During the intervention period, multiplex molecular testing was performed on all children with results available to clinicians. Primary outcome was return visits to a health care provider within 10 days of enrollment.
Results Potential pathogens were identified by clinician ordered tests in 19/571 (3.3%) in the pre-intervention period compared to 434/586 (74%) in the intervention period; clinically relevant pathogens were detected in 2.1% and 15% respectively. In the multivariate model adjusting for potential confounders, the intervention was associated with a 21% reduction in the odds of any return visit (OR 0.79; 95% CI 0.70-0.90). Appropriate treatment was prescribed in 11.3% compared to 19.6% during the intervention period(P=0.22).
Conclusions Routine molecular multiplex testing for all children presenting to the ED with AGE detected more clinically relevant pathogens and led to a 21% decrease in return visits. Additional research is needed to define patients most likely to benefit from testing.
INTRODUCTION
Acute gastroenteritis is a major cause of emergency department (ED) visits and hospitalization in countries with advanced economies and a major cause of death and disability worldwide.[1, 2] For many children with gastroenteritis, rehydration, antiemetics, and reassurance are the only therapy needed. However, a range of bacterial and protozoal infections may require definitive diagnosis and targeted therapy.[3, 4] Multiplex molecular diagnostic panels available in recent years greatly enhance detection of a wide range of pathogens.[5-10] The potential benefits of rapid and accurate detection include decreased health care utilization, decreased parental anxiety, appropriate antimicrobial therapy, and improved infection control and outbreak recognition.[11] The potential negative consequences include overtreatment of incidentally detected organisms, increased cost, and the potential failure to provide isolates to public health laboratories for molecular characterization.[12]
Relatively few studies to date have prospectively evaluated the impact of multiplex molecular testing on improving clinical and patient-centered outcomes. We conducted a multicenter prospective pre-post intervention study to evaluate the direct and indirect impact of the introduction of a multiplex molecular diagnostic panel on care for children presenting to pediatric Emergency Departments (EDs) with acute gastroenteritis.
METHODS
Design
The objective of the FilmArray GI Panel IMPACT (Implementation of a Molecular Diagnostic for Pediatric Acute Gastroenteritis) study was to determine the clinical impact of multiplex molecular diagnostic testing for stool pathogens versus clinician-selected diagnostic testing in children presenting to EDs with acute gastroenteritis. We conducted a prospective, multicenter pragmatic-stepped wedge study at five academic children’s hospitals. (Supplemental Figure 1) Children were enrolled between April 2015 and September 2016. Enrollment was staggered in a stepped-wedge fashion with the goal of reducing the potential impact of seasonal variation on the etiology of diarrhea. The start of the study, however, was impacted based on each institution’s timeline for completing study. The study was approved by the institutional review board at each institution.
Time course of enrollment in Pre-intervention and Intervention periods by study site
Eligibility
Children <18 years presenting to the ED or on-site urgent care center were eligible if they had symptoms of gastroenteritis (diarrhea or nausea/vomiting as a chief complaint) for at least 24 hours. Children were excluded if symptoms were present <24 hours or ≥14 days, if a diagnosis other than gastroenteritis was apparent (e.g. appendicitis, inflammatory bowel disease), if they or a family member had been previously enrolled in the study, or if informed consent could not be obtained.
Study procedures and intervention
Written informed consent was obtained from parents or legal guardians; children provided assent as age-appropriate. Trained study personnel interviewed subjects using a standardized questionnaire at enrollment. The questionnaire included demographic information, symptoms, epidemiologic exposures and previous treatment. Study personnel interviewed subjects again by telephone 7-10 days after enrollment to determine duration of illness, subsequent medical visits, tests or treatment, secondary illness in the family, and impact on family members. Medical charts were systematically abstracted after the final contact to collect laboratory tests ordered, results, treatments prescribed, hospital admission and return visits.
All children were asked to provide a stool specimen during the visit or within 48 hours for multiplex PCR testing, regardless of whether the testing was ordered for clinical purposes. Specimens were placed in Cary-Blair transport medium. During the pre-intervention period, diagnostic testing was at the discretion of the clinicians in the ED. Stool specimens were stored and later tested by multiplex PCR, but results were not available to the clinicians or patients. After roughly 100 evaluable patients were enrolled in the pre-intervention period, each site prepared to transition to the intervention period.
Before initiating the intervention period, clinicians in each ED were educated about the multiplex PCR panel, including performance characteristics, clinical features of each pathogen, recommended management, and explanations in patient-friendly language (http://cstockmann.github.io/IMPACT/index.html). Stool samples from children enrolled in the intervention period were tested by multiplex PCR in real-time at the hospital laboratory (with results generally available within ∼1-4 hours of receipt) and communicated to the ED. If the patient had been discharged before results were available, clinicians contacted the family by the next morning. Clinicians were free to order additional testing at their discretion.
During the pre-intervention period, children were included in the analysis regardless of whether a specimen was provided for multiplex PCR; during the intervention period, children were excluded if they could not provide a stool specimen within 48 hours of enrollment.
Laboratory testing
Clinician-selected testing was performed in the hospital laboratory by standard methods and included stool culture, Clostridiodes difficile testing by EIA or nucleic acid amplification, rotavirus and adenovirus antigen detection by EIA, norovirus detection by nucleic acid amplification, or protozoal detection by microscopy or EIA for Giardia lamblia/duodenalis and Cryptosporidium. We used the BIOFIRE FILMARRAY Gastrointestinal (GI) Panel assay for multiplex PCR testing (bioMérieux, Salt Lake City, UT). This platform detects 22 pathogens. (Supplemental Table 1) [5] During the intervention period, when pathogens of public health significance were detected by PCR (Campylobacter spp., Salmonella spp., Shiga toxin-producing Escherichia coli [STEC],STEC, Shigella spp./EIEC), specimens were reflexed to culture and/or sent to Public Health laboratories.
Pathogens detected by multiplex PCR and those considered “treatable” or “clinically actionable,” defined as either treatable or the use of antibiotics would be potentially harmful
Outcome measures
The primary outcome was return visits to a health care provider within 10 days of enrollment. Secondary outcomes included number of return visits, return visit to ED or hospitalization, number of pathogens detected, number of potentially treatable pathogens detected (Campylobacter, Shigella, Vibrio sp., Yersinia, Plesiomonas, Cryptosporidium, Cyclospora cayetanensis, Entamoeba histolytica and G. lamblia), clinically relevant pathogens detected – e.g. those that might alter care (potentially treatable pathogens plus those for which withholding antibiotics is important, i.e. Salmonella and STEC), and the proportion of children receiving appropriate treatment. Since the clinical significance of C. difficile detection by PCR alone in outpatients is challenging to interpret, we reported total detections and detection of C. difficile as the sole pathogen in children >2 years of age but did not define it as a potentially treatable pathogen. We also measured the impact on secondary illness in family members, absence from daycare, school, and work, parental response to knowing the diagnosis, and detection of outbreaks.
Statistical analysis
Power calculations were based on historical data on return visits for patients with gastroenteritis seen at Primary Children’s Hospital showing a mean of 0.5 additional healthcare encounters per patient within seven days of the initial encounter. Sample size was limited by funding to 1100, providing 70% power to detect a 25% reduction in return visits with an alpha of 0.05.
Patient characteristics, clinical findings, tests ordered, pathogens detected, and outcomes were summarized using descriptive statistics for pre and post-intervention periods and compared using chi-squared or Mann-Whitney Wilcoxon tests for categorical or continuous data. Differences were presented as odds ratios and exact confidence intervals. To account for imbalances in the study periods, the primary analyses of the impact of multiplex molecular diagnostic testing on return visits (any return, ED return or hospitalization, and number of return visits within 10 days of enrollment) was based on logistic or negative binomial generalized estimating equations models, as appropriate, accounting for within site clustering, and adjusted for patient age, insurance, illness duration, pathogen type, gender, race, ethnicity, diarrhea, fever, constipation, number of stools in last 24 hours, and quarter by year.[13, 14] The consistency of the estimated impact of multiplex molecular diagnostic testing across subpopulations of interest defined by site, gender, race, patient age, and illness duration was assessed via subgroup analyses mirroring the full analysis. To assess the impact of a Shigella outbreak in Kansas City during the time of study, we conducted sensitivity analyses omitting Children’s Mercy Hospital (CMH) observations. [15]
RESULTS
Demographics
Between April 2015 and September 2016, we enrolled 1,157 patients (571 in the pre-intervention period and 586 in the intervention period). (Figure 1) The mean age was 4.9 years; 64% were under 5 years of age. (Table 1) Race/ethnicity and season of enrollment differed between the pre-intervention and intervention periods as a result of the staggered enrollment by site. (Table 1 and supplemental Figure 1). The majority of children presented with vomiting and diarrhea; diarrhea was bloody in 13% (Table 2). More children in the intervention period had diarrhea and more had fever in the pre-intervention period. Symptoms were present for a median of 2 days (IQR: 1-5) before presentation.
Consort diagram of subject disposition
Microbiology
During the pre-intervention period, clinicians ordered etiologic testing on 80 (14%) of 571 patients. (Table 2). Diagnostic tests were more likely to be ordered if the stool was reported as bloody (33/52 [64%]) than non-bloody (43/402 [11%]; Odds Ratio [OR] 14.0, 95% Confidence Intervals [CI] 7.6-27.7). Testing was not associated with reported fever; tests were obtained in 40/324 (12.4%) children with fever and 38/239 (15.9%) without fever (OR 0.75; 95% CI 0.46-1.20).
Clinician-ordered testing detected a pathogen in 19 (3.3%) of 571 patients during the pre-intervention period, including 14 (2.5%) with a treatable pathogen and 16 (2.8%) with a clinically relevant pathogen. (Table 3) Stool was available for batched testing by multiplex testing in 375/571 patients during the pre-intervention period. A total of 383 potential pathogens were detected in 262 patients (70%). Viral pathogens were detected in 135 (36%); EPEC, norovirus, C. difficile, and Shigella/EIEC were the most commonly detected pathogens overall. C. difficile was detected in 43 (11.5%) of 375 children by molecular testing, however, C. difficile was the sole pathogen in only 8 children ≥2 years of age.
During Pre-intervention period, multiplex PCR testing was performed on stored stool specimens and not available to clinicians. During Intervention period multiplex testing was performed in real-time and available to clinicians.
During the pre-intervention period, potentially treatable pathogens were detected by multiplex molecular testing in 65 (17.3%) of 375 patients tested compared to 12 (2.1%) of 571 children by clinician ordered testing (difference 14.3%, 95% CI 10.3%-18.2%; P<0.001). Clinically relevant pathogens were detected in 84 (22.4%) by multiplex molecular testing compared to 14 (2.5%) by clinician-ordered testing (difference 18.6%; 95% CI 14.1%-23.0%; P<0.001).
During the intervention period, 627 potential pathogens were detected by multiplex molecular tesing in 434 (74%) of 571 children. Norovirus, C. difficile, EPEC and sapovirus were the most common pathogens detected. Only 23 children (3.9%) with C. difficile were ≥2 years of age and had no other pathogens detected. Viral pathogens were detected in 294 patients (50%) during the intervention period, significantly more than during the pre-intervention period (36%; P<0.001). Shigella spp./EIEC was detected less frequently during the intervention period (24 [4.1%] vs 33 [8.8%]; P=0.003). This was driven by a community-wide outbreak of Shigella in Kansas City impacting CMH during the pre-intervention period. Multiplex PCR results were available and shared with the family before leaving the ED in 141 (24%) cases.
Outcomes
The likelihood of follow-up visits was not significantly lower during the intervention period in univariate analysis. During the intervention period 186 (32%) children had at least one follow-up visit with a health care provider after the enrollment visit to the ED compared to 174 (30%) during the pre-intervention period (OR 1.06 95% CI 0.82-1.37). Of these 54 (9.2%) were hospitalized or returned to urgent care or the ED during the intervention period, compared to 53 children (9.3%) during the pre-intervention period (OR 1.00; 95% CI 0.66-1.53).
During intervention period, children with a viral pathogen detected were less likely to have a return visit (75 of 294 [26%]) compared to those without a virus detected (111 0f 292 [38%], OR 0.56; 95% CI 0.39-0.79; P = 0.001). This was not the case during the pre-intervention period (39 of 135 [29%] vs 72 of 240 [30%]; OR 0.95 95% CI 0.60-1.51). During the intervention period children who were enrolled in winter were less likely to have a repeat visit (62 of 217 [29%] compared to 21 of 44 [48%]; OR 0.44; 95% CI 0.23-.85; P= 0.01) during the pre-intervention period. Conversely, during the intervention period those enrolled during summer were more likely to have a return visit (28 of 66 [42%] vs. 86 of 305 [28%]; OR 1.9; 95% CI 1.08-3.2; P = 0.023).
In the multivariate model adjusting for age, insurance, illness duration, pathogen type, sex, race, ethnicity, diarrhea, fever, number of stools, and quarter by year of patient presentation, the intervention was associated with a 21% reduction in the odds of any return visit (OR 0.79; 95% CI 0.70-0.90; P=<0.001; Table 4). Multivariate results for the number of return visits were similar (rate ratio 0.83; 95% CI 0.70-0.97; P=0.023; Table 4). However, multivariate analyses did not show a statistically significant difference between pre- and post-intervention periods in ED visits or hospitalizations (OR 0.93; 95% CI 0.57-1.51; p=0.771).
Multivariate sensitivity analyses omitting CMH to account for the Shigella spp. outbreak yielded similar results for any return visit (OR 0.74; 95% CI 0.68-0.80; P<0.001; Supplemental Table 1; number of return visits rate ratio 0.77; 95% CI 0.72-0.83; P<0.001). In the sensitivity analysis the odds of return to the ED or hospitalization was reduced during the intervention period (OR 0.70; 95% CI 0.56-0.86; P=0.001).
Results of multivariate analyses of any return visit within subgroups were broadly consistent with the main analysis (Figure 2). Analyses restricted to individual sites had substantial uncertainty due to collinearity between intervention periods and time within site. The sensitivity analysis omitting observations from CMH because of the Shigella spp. outbreak were similar to the main analysis with slighty greater point estimates of the reduction in rate of return visits. (Supplemental table 2) There was no significant difference between the pre- and intervention periods in the proportion of children who received an antibiotic in the ED (3.5% vs 4.1%) or after discharge (4.2% vs 3.8%) in univariate or multivariate analysis. (Table 3) However, numerically more patients received the appropriate treatment for a potentially treatable pathogen during the intervention period with multiplex molecular testing; 12 (19.6%) of 61 vs 7 (11.3%) of 62 (P=0.22) and appropriate treatment for Shigella; 11 (46%) of 24 vs 7 (21%) of 33 (P = 0.08).
Forest plot of subgroup analyses for any return visits.*
Impact of illness on the family was notable. Half of caregivers in both periods reported missing work (median 2 days; IQR 1-3.5). Diarrhea developed in one or more family contacts after the child was enrolled in 30% of cases. These factors did not differ by intervention period. Of 413 caregivers in the intervention arm who reported learning the etiology of their child’s illness in the intervention period, 215 (52%) reported they were “very comfortable” caring for their child’s illness compared to 304 (53%) of caregivers in the pre-intervention period.
DISCUSSION
Recent technological advances have resulted in the availability of rapid, sensitive, and comprehensive molecular diagnostics for diarrheal illness.[7, 8]. Compared with the wealth of evidence supporting rehydration for pediatric gastroenteritis management, there is a paucity of studies demonstrating impact of these diagnostics on health and patient-centered outcomes, their cost effectiveness, or when they are most useful. [7, 16, 17] In this multi-center prospective pragmatic study, we found that among children presenting to EDs with gastroenteritis, the use of a multiplex molecular diagnostic was associated with a 21% reduction in the likelihood of return visits to healthcare compared with clinician-selected diagnostic testing. As in other studies, we found that multiplex molecular testing substantially increased detection of potential pathogens, including treatable pathogens and those that warranted withholding of empiric antibiotics. Awareness of diagnostic testing results was not associated with increased caregiver comfort in caring for the child’s illness.
We prospectively compared clinician-, patient-, and family-based outcomes before and after introduction of a multiplex molecular diagnostic for children presenting to an ED with gastroenteritis. The pre-intervention and intervention periods were not balanced for a number of key variables associated with diarrheal disease, and our unadjusted analysis showed no differences in likelihood of follow-up visits. After adjusting for factors including demographics, pathogen type, and season, we found a 21% reduction in odds of any return visits to healthcare (OR 0.79; 95% CI 0.70-0.90), our primary endpoint. Stratified analyses suggested the effect was most pronounced in children with a viral pathogen detected (OR 0.56; 95% CI 0.39-0.79), and children enrolled in the winter (OR 0.44; 95% CI 0.23-0.85), the season with the highest rates of viral gastroenteritis.[18] Knowledge of a virus as cause of illness may have reduced clinician and patient uncertainty about the clinical course and need for re-evaluation.
Advantages of molecular diagnostics over traditional stool culture include increased sensitivity for pathogen detection and faster turnaround time, potentially resulting in improvements in appropriate use of antibiotics. We found that multiplex molecular testing of all patients identified a potentially treatable pathogen in a higher proportion (17.3%) of patients during the pre-intervention period compared to clinician-ordered testing (3.2%) as well as clinically relevant pathogens (22% vs 2.8%). We did not detect a significant improvement in overall appropriate antibiotic prescribing among children with a treatable pathogen, although consistent with other studies, fewer than 20% of children had a treatable pathogen in either period.[9, 19, 20] We observed an increase in percentage of patients with Shigella spp./EIEC who received appropriate antibiotics during the intervention period although this was not statistically significant (p = 0.08). Knowledge of the outbreak in the pre-intervention period may have increased appropriate empiric therapy, and therefore reduced the impact of the intervention. Recent observational studies have shown that introduction of a multiplex GI PCR panel was associated with improvements in the selection and duration of antimicrobial therapy [7, 10] [21] [22] [23]
The burden of pediatric gastroenteritis extends beyond the patient, and includes significant nonmedical costs, the largest of which are foregone earnings due to missed work and transportation costs. Half of caregivers reported missing work, and approximately 30% of household contacts subsequently developed diarrhea. Although we did not observe differences in days of work missed with the intervention, it is likely that the decrease in return visits translates to economic benefits. Cost savings attributable to a multiplex diagnostic panel has been shown in an inpatient population [16, 21, 22] and in an a study of adult based on claims data[24], but not to our knowledge among pediatric ED patients. Future studies are warranted to examine the cost impacts and cost-effectiveness.
Further studies are needed to identify the types of patients most likely to benefit from molecular diagnostics, as opposed to the “test all” strategy used in this study. Unfortunately, existing recommendations for selecting patients most likely to have bacterial infection do not perform optimally.[25] Novel strategies for stewardship of molecular diagnostics are needed[26, 27], including the development of clinical decision support tools.[28, 29]
The patterns of organisms detected found in our study is consistent with prior studies using multiplex detection for children with diarrhea in high-income countries. We found EPEC, norovirus, and C. difficile to be the most commonly detected pathogens overall, similar to previous studies.[9, 20] However, interpreting the results of molecular detection of potential pathogens can be challenging, since not all molecular detections correspond to the etiology of symptoms due to detection of asymptomatic carriage or lack of specificity of the target.[30] This is particularly problematic for EPEC and C. difficile. We did not observe increased prescription of drugs for C. difficile, despite increased detection. EPEC was commonly detected (>15% of samples), but the clinical interpretation of detection of the single gene target (eae) is unknown. Case-control studies using molecular diagnostics, such as ones already conducted in other settings [31] [32] are urgently needed in the US to determine the relevance of organisms detected through various gene targets. Quantitative PCR thresholds from case-control studies may increase specificity. [33]
Our study has limitations. First, despite the stepped-wedge design of the enrollment across sites, none of the sites were able to capture the same months during the pre- and post-intervention period. Thus, our analysis required adjustment by season and time of year. Second, a Shigella outbreak occurred at one site during the study[15], which led to a higher proportion of Shigella detected than would be expected. However, our sensitivity analyses excluding this site confirmed the results. Third, only 24% of children were able to get results before discharge from the ED because of failure to produce a specimen. Use of rectal swabs might improve this and increase the impact of diagnosis.[34] Fourth, power was modest. Lastly, a larger proportion (15% vs. 4%) of those enrolled in the pre-intervention phase were lost to follow-up compared to the intervention phase.
In conclusion, in a multi-center prospective pragmatic study of a multi-pathogen molecular diagnostic panel for pediatric gastroenteritis, we found that use of panel improved pathogen detection and decreased likelihood of return visits to healthcare. Further research is needed to identify patients most likely to benefit from the use of these tests and to optimize the cost effectiveness of different diagnostic approaches to pediatric gastroenteritis.
ACKNOWLEDGEMENTS
We thank Julian Dorsch, Tania Baca, and the ED physicians, pharmacists and research coordinators who made this study possible. We are indebted to the families who participated.
Footnotes
Registration: ClinicalTrials.gov identifier NCT02248285
Funding: This work was supported by National Institute of Allergy and Immunology at the National Institutes of Health [grant number R01AI104593] to KMB; additional funding from BioFire Diagnostics (now bioMérieux)
Disclosures:
REFERENCES
Subject Area
Reviews and Context
0
Comment
0
TRIP Peer Reviews
0
Community Reviews
0
Automated Services
0
Blogs/Media
Author Videos