Parallel Reinforcement Learning and Gaussian Process Regression for Improved Physics-Based Nasal Surgery Planning

Lecture notes in computer science, pp. 79–96

Abstract

Abstract Septoplasty and turbinectomy are among the most frequent but also most debated interventions in the field of rhinology. A previously developed tool enhances surgery planning by physical aspects of respiration, i.e., for the first time a reinforcement learning (RL) algorithm is combined with large-scale computational fluid dynamics (CFD) simulations to plan anti-obstructive surgery. In the current study, an improvement of the tool’s predictive capabilities is investigated for the aforementioned types of surgeries by considering two approaches: (i) training of parallel environments is executed on multiple ranks and the agents of each environment share their experience in a pre-defined interval and (ii) for some of the state-reward combinations the CFD solver is replaced by a Gaussian process regression (GPR) model for an improved computational efficiency. It is found that employing a parallel RL algorithm improves the reliability of the surgery planning tool in finding the global optimum. However, parallel training leads to a larger number of state-reward combinations that need to be computed by the CFD solver. This overhead is compensated by replacing some of the computations with the GPR algorithm, i.e., around $$6\%$$ 6 % of the computations can be saved without significantly degrading the predictions’ accuracy. Nevertheless, increasing the number of state-reward combinations predicted by the GPR algorithm only works to a certain extent, since this also leads to larger errors.

Authors 4

1. Mario Rüttgers corresponding

   Forschungszentrum Jülich · Jülich Supercomputing Centre · Kobe University

   Affiliation as printed

   Department of Computational Science, Graduate School of System Informatics, Kobe University, 657-8501, Kobe, Japan

   Jülich Supercomputing Centre, Forschungszentrum Jülich, 52428, Jülich, Germany

2. Fabian Hübenthal Aachen Institute of Aerodynamics Chair of Fluid Mechanics

   Kobe University · RWTH Aachen University

   Affiliation as printed

   Department of Computational Science, Graduate School of System Informatics, Kobe University, 657-8501, Kobe, Japan

   Institute of Aerodynamics and Chair of Fluid Mechanics (AIA), RWTH Aachen University, 52062, Aachen, Germany

3. Makoto Tsubokura

   RIKEN Center for Computational Science · Kobe University

   Affiliation as printed

   Complex Phenomena Unified Simulation Research Team, RIKEN Center for Computational Science, 650-0047, Kobe, Japan

   Department of Computational Science, Graduate School of System Informatics, Kobe University, 657-8501, Kobe, Japan

4. Andreas Lintermann

   Forschungszentrum Jülich · Jülich Supercomputing Centre · Kobe University

   Affiliation as printed

   Department of Computational Science, Graduate School of System Informatics, Kobe University, 657-8501, Kobe, Japan

   Jülich Supercomputing Centre, Forschungszentrum Jülich, 52428, Jülich, Germany