A novel 3D image registration technique for augmented reality vision in minimally invasive thoracoscopic pulmonary segmentectomy
International Journal of Computer Assisted Radiology and Surgery, vol. 20, pp. 787–795
Abstract
PURPOSE: In this feasibility study, we aimed to create a dedicated pulmonary augmented reality (AR) workflow to enable a semi-automated intraoperative overlay of the pulmonary anatomy during video-assisted thoracoscopic surgery (VATS) or robot-assisted thoracoscopic surgery (RATS). METHODS: Initially, the stereoscopic cameras were calibrated to obtain the intrinsic camera parameters. Intraoperatively, stereoscopic images were recorded and a 3D point cloud was generated from these images. By manually selecting the bifurcation key points, the 3D segmentation (from the diagnostic CT scan) was registered onto the intraoperative 3D point cloud. RESULTS: Image reprojection errors were 0.34 and 0.22 pixels for the VATS and RATS cameras, respectively. We created disparity maps and point clouds for all eight patients. Time for creation of the 3D AR overlay was 5 min. Validation of the point clouds was performed, resulting in a median absolute error of 0.20 mm [IQR 0.10-0.54]. We were able to visualize the AR overlay and identify the arterial bifurcations adequately for five patients. In addition to creating AR overlays of the visible or invisible structures intraoperatively, we successfully visualized branch labels and altered the transparency of the overlays. CONCLUSION: An algorithm was developed transforming the operative field into a 3D point cloud surface. This allowed for an accurate registration and visualization of preoperative 3D models. Using this system, surgeons can navigate through the patient's anatomy intraoperatively, especially during crucial moments, by visualizing otherwise invisible structures. This proposed registration method lays the groundwork for automated intraoperative AR navigation during minimally invasive pulmonary resections.
Authors 7
-
Affiliation as printed
Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
-
Delft University of Technology
Affiliation as printed
Computer Vision Lab, TU Delft, Delft, The Netherlands
-
Delft University of Technology
Affiliation as printed
Computer Graphics and Visualization Lab, TU Delft, Delft, The Netherlands
-
Leiden University Medical Center
Affiliation as printed
Department of Cardiothoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands
-
Erasmus MC · University Medical Center Utrecht
Affiliation as printed
Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
Department of Cardiothoracic Surgery, University Medical Center Utrecht, Utrecht, The Netherlands
-
Affiliation as printed
Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
-
Leiden University Medical Center · Erasmus MC
Affiliation as printed
Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands. e.a.f.mahtab@lumc.nl
Department of Cardiothoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands. e.a.f.mahtab@lumc.nl
Department of Cardiothoracic Surgery, Erasmus University Medical Center, Rotterdam, The Netherlands
Department of Cardiothoracic Surgery, Leiden University Medical Center, Leiden, The Netherlands
Cited by 12 stored of 12
12 results
No patents citing this paper on Lens.org (checked 2026-10-11).