Abstract
Background Algorithmic decision making (ADM) utilizes algorithms to collect and process data and develop models to make or support decisions. Advances in artificial intelligence (AI) have led to the development of support systems that can be superior to medical professionals without AI support in certain tasks. However, whether patients can benefit from this remains unclear. The aim of this systematic review is to assess the current evidence on patient-relevant benefits and harms when healthcare professionals use ADM systems (developed using or working with AI) compared to healthcare professionals without AI-related ADM (standard care) - regardless of the clinical issues. Furthermore, for interpreting collected evidence and analysing preconditions for the implementation of AI-related ADM in healthcare, experts from research, practice, and regulation will be interviewed.
Methods Following the PRISMA statement and the MECIR standards for reporting systematic reviews, MEDLINE and PubMed (via PubMed), EMBASE (via Elsevier), IEEE Xplore, CENTRAL will be searched using English free text terms in title/abstract, Medical Subject Headings (MeSH) terms and Embase Subject Headings (Emtree) fields. Additional studies will be identified by contacting authors of included studies and through reference lists of included studies. Grey literature searches will be conducted in Google Scholar. Risk of bias will be assessed by using Cochrane’s RoB 2 for randomised trials and ROBINS-I for non-randomised trials. Transparent reporting of the included studies will be assessed using the CONSORT-AI extension statement. Following the SRQR statement, semi-structured interviews will be conducted and analysed with the help of a qualitative content analysis according to Mayring. Based on the research questions and the findings of the systematic review, the study and interview guide will be developed a priori.
Discussion It is expected that there will be a substantial shortage of suitable studies that compare healthcare professionals with and without ADM systems concerning patient-relevant endpoints. This can be attributed to the prioritization of technical quality criteria and, in some cases, clinical parameters over patient-relevant endpoints in the development of study designs. Furthermore, it is anticipated that a significant portion of the identified studies will exhibit relatively poor methodological quality and provide only limited generalizable results.
Systematic review registration This study is registered within Prospero (CRD42023412156).
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
This study did not receive any funding.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
Ethics committee/IRB of University of Potsdam, Am Neuen Palais 10, 14469 Potsdam, Germany gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
List of abbreviations
- ADM
- Algorithmic Decision Making
- AI
- Artificial Intelligence
- AUROC
- Area Under the Receiver Operating Characteristic Curve
- CENTRAL
- Cochrane Central Register of Controlled Trials
- CNN
- Convolutional Neural Network
- CONSORT
- Consolidated Standards of Reporting Trials
- CONSORT-AI
- Consolidated Standards of Reporting Trials for Artificial Intelligence
- CRD
- Centre for Reviews and Dissemination
- EMTREE
- Embase subject headings
- HR
- Heart Rate
- ICU
- Intensive Care Unit
- IEEE
- Institute of Electrical and Electronics Engineers
- IQWiG
- German Institute for Quality and Efficiency in Healthcare
- MECIR
- Methodological Expectations of Cochrane Intervention Reviews
- MeSH
- Medical Subject Headings
- ML
- Machine Learning
- nRCT
- non Randomized Controlled Trial
- PICO
- Participants, Intervention, Control, Outcome
- PRISMA
- Preferred Reporting Items for Systematic Review and Meta-Analysis
- PRISMA-P
- Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols
- PROSPERO
- International Prospective Register of Systematic Reviews
- RCT
- Randomized Controlled Trial
- ResNet-18
- A convolutional neural network that is 18 layers deep
- RoB 2
- Revised Cochrane Risk-of-Bias Tool for randomized trials
- ROBINS-I
- Risk-of-Bias in non-randomized studies for interventions
- RR
- Respiratory Rate
- SD
- Standard Deviation
- SpO2
- Oxygen Saturation
- SRQR
- Standards for Reporting Qualitative Research
