A Baseline Solution for the ISBI 2024 Dreaming Challenge

IEEE International Symposium on Biomedical Imaging (ISBI), pp. 1–3

Abstract

Diminished Reality is a technique for the removal of objects from the surroundings, providing a better view of otherwise obstructed features. This work explores the application of Diminished Reality in a medical setting, specifically aiming to visually eliminate surgical tools from operation sites for improved visibility and inspection. To this end, we introduce the Diminished Reality for Emerging Applications in Medicine through Inpainting (DREAMING) challenge. For the challenge, we created a dataset of simulated surgery scenes with obstructions from surgical instruments and hands. We evaluate an existing video inpainting model to generate diminished images from our dataset to set a baseline for the challenge. We show that the off-the-shelf model already delivers satisfactory results across some conditions, but several limitations prohibit its application for the proposed real-world scenario.

Authors 7

1. Timo van Meegdenburg

   Essen University Hospital

   Affiliation as printed

   University Hospital Essen (AöR),Institute for Artificial Intelligence in Medicine

2. Gijs Luijten

   Essen University Hospital

   Affiliation as printed

   University Hospital Essen (AöR),Institute for Artificial Intelligence in Medicine

3. Jens Kleesiek

   Essen University Hospital

   Affiliation as printed

   University Hospital Essen (AöR),Institute for Artificial Intelligence in Medicine

4. Behrus Puladi Aachen Department of Oral and Maxillofacial Surgery

   Universitätsklinikum Aachen

   Affiliation as printed

   University Hospital Aachen,Department of Oral and Maxillofacial Surgery

5. André Filipe Ferreira

   University of Minho

   Affiliation as printed

   University of Minho,Center Algoritmi / LASI,Braga,Portugal

6. Jan Egger

   Essen University Hospital

   Affiliation as printed

   University Hospital Essen (AöR),Institute for Artificial Intelligence in Medicine

7. Christina Gsaxner

   Essen University Hospital

   Affiliation as printed

   University Hospital Essen (AöR),Institute for Artificial Intelligence in Medicine