Improving instrument detection for a robotic scrub nurse using multi-view voting

verfasst von

Jorge Badilla-Solórzano, Sontje Ihler, Nils Claudius Gellrich, Simon Spalthoff

Abstract

Purpose: A basic task of a robotic scrub nurse is surgical instrument detection. Deep learning techniques could potentially address this task; nevertheless, their performance is subject to some degree of error, which could render them unsuitable for real-world applications. In this work, we aim to demonstrate how the combination of a trained instrument detector with an instance-based voting scheme that considers several frames and viewpoints is enough to guarantee a strong improvement in the instrument detection task. Methods: We exploit the typical setup of a robotic scrub nurse to collect RGB data and point clouds from different viewpoints. Using trained Mask R-CNN models, we obtain predictions from each view. We propose a multi-view voting scheme based on predicted instances that combines the gathered data and predictions to produce a reliable map of the location of the instruments in the scene. Results: Our approach reduces the number of errors by more than 82% compared with the single-view case. On average, the data from five viewpoints are sufficient to infer the correct instrument arrangement with our best model. Conclusion: Our approach can drastically improve an instrument detector’s performance. Our method is practical and can be applied during an actual medical procedure without negatively affecting the surgical workflow. Our implementation and data are made available for the scientific community (https://github.com/Jorebs/Multi-view-Voting-Scheme).

Organisationseinheit(en)

Institut für Mechatronische Systeme

Externe Organisation(en)

Medizinische Hochschule Hannover (MHH)

Typ

Artikel

Journal

International journal of computer assisted radiology and surgery

Band

18

Seiten

1961-1968

Anzahl der Seiten

8

ISSN

1861-6410

Publikationsdatum

11.2023

Publikationsstatus

Veröffentlicht

Peer-reviewed

Ja

ASJC Scopus Sachgebiete

Chirurgie, Biomedizintechnik, Radiologie, Nuklearmedizin und Bildgebung, Maschinelles Sehen und Mustererkennung, Angewandte Informatik, Gesundheitsinformatik, Computergrafik und computergestütztes Design

Elektronische Version(en)

https://doi.org/10.1007/s11548-023-03002-0 (Zugang: Offen)