Abstract
Background Lack of anatomy recognition represents a clinically relevant risk factor in abdominal surgery. While machine learning methods have the potential to aid in recognition of visible patterns and structures, limited availability and diversity of (annotated) laparoscopic image data restrict the clinical potential of such applications in practice. This study explores the potential of machine learning algorithms to identify and delineate abdominal organs and anatomical structures using a robust and comprehensive dataset, and compares algorithm performance to that of humans.
Methods Based on the Dresden Surgical Anatomy Dataset providing 13195 laparoscopic images with pixel-wise segmentations of eleven anatomical structures, two machine learning algorithms were developed: individual segmentation algorithms for each structure, and a combined algorithm with a common encoder and structure-specific decoders. Performance was assessed using F1 score, Intersection-over-Union (IoU), precision, recall, and specificity. Using the example of pancreas segmentation on a sample dataset of 35 images, algorithm performance was compared to that of a cohort of 28 physicians, medical students, and medical laypersons.
Results Mean IoU for segmentation of intraabdominal structures ranged from 0.28 to 0.83 and from 0.32 to 0.81 for the structure-specific and the combined semantic segmentation model, respectively. Average inference for the structure-specific (one anatomical structure) and the combined model (eleven anatomical structures) took 20 ms and 54 ms, respectively. The structure-specific model performed equal to or better than 27 out of 28 human participants in pancreas segmentation.
Conclusions Machine learning methods have the potential to provide relevant assistance in anatomy recognition in minimally-invasive surgery in near-real-time. Future research should investigate the educational value and subsequent clinical impact of respective assistance systems.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
FRK, SL, JW, and SS were supported through project funding within the Else Kroener Fresenius Center for Digital Health (EKFZ), Dresden, Germany (project CoBot). FRK received funding from the Medical Faculty of the Technical University Dresden within the MedDrive Start program (grant number 60487) and from the Joachim Herz Foundation (Add-On Fellowship for Interdisciplinary Life Science). FMR received a doctoral student scholarship from the Carus Promotionskolleg Dresden.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
All experiments were performed in accordance with the ethical standards of the Declaration of Helsinki and its later amendments. The local Institutional Review Board (ethics committee at the Technical University Dresden) reviewed and approved this study (approval number: BO-EK-137042018). The trial was registered on clinicaltrials.gov (trial registration ID: NCT05268432). Written informed consent to laparoscopic image data acquisition, data annotation, data analysis, and anonymized data publication was obtained from all participants. Before publication, all data was anonymized according to the general data protection regulation of the European Union.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines and uploaded the relevant EQUATOR Network research reporting checklist(s) and other pertinent material as supplementary files, if applicable.
Yes
Data Availability
The Dresden Surgical Anatomy Dataset is publicly available via the following link: https://figshare.com/s/d7a60b74989a9cab2f7f. All other data generated and analyzed during the current study are available from the corresponding authors on reasonable request. To gain access, data requestors will need to sign a data access agreement.
Abbreviations
- AI
- Artificial Intelligence
- IoU
- Intersection-over-Union
- SD
- Standard deviation
