ABSTRACT
Objectives Conduct a systematic review of the existing evidence base pertaining to the conduct of randomised controlled trials of clinical decision support systems embedded within electronic health record systems. Further, to describe whether key features of trial design and implementation were consistently reported.
Materials and Methods A systematic search of MEDLINE was conducted in April 2022. Three independent reviewers screened the search results. A 27-item checklist was used to extract data from the screened studies. A subgroup analysis was conducted to classify trials of clinical decision support systems based on whether they encouraged guideline adherence or represented new knowledge generating mechanisms.
Results 5,213 records were retrieved. Following screening, 106 studies were included in the review. The majority of studies evaluated active alerts seeking to improve adherence to clinical guidelines rather than generate new knowledge. Few studies quantified the existing ecosystem of decision support at the study site, or explored phenomena like alert fatigue.
Discussion This systematic review provides a detailed analysis of the characteristics of trials evaluating clinical decision support systems. It highlights significant under-reporting of key factors which may affect the reproducibility and generalisability of trial results - particularly with respect to measurement of alert fatigue, description of the underlying digital ecosystem and additional co-interventions used within trials.
Conclusion As clinical workflows undergo digital transformation, randomised controlled trials of clinical decision support systems require greater standardisation, in both conduct and reporting. This represents an area of expanding interest given the increasing use of artificial intelligence-enabled decision support.
STRENGTHS & LIMITATIONS
- This study presents the results of an updated systematic review of studies evaluating the effectiveness clinical decision support systems.
- It used a comprehensive checklist to extract detail pertaining to five information domains on trial quality and description.
- Studies were evaluated to determine whether the clinical decision support system was knowledge generating or designed to improve guideline adherence.
- The review was limited to randomised trials and excluded quasi-experimental and observational studies of clinical decision support systems.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study is supported by the NIHR Central London Patient Safety Research Collaboration (CL PSRC), reference number NIHR204297. The views expressed are those of the authors and not necessarily those of the NIHR or the Department of Health and Social Care. MGW is supported by the NIHR Biomedical Research Centre and Patient Safety Research Collaborative at UCL/H. YC is supported by the NIHR Biomedical Research Centre at UCLH. MN is supported by the UKRI CDT in AI for Healthcare, http://ai4health.io (EP/S023283/1).
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
Data extraction tables available on request.
