On Controller Tuning with Time-Varying Bayesian Optimization

Proceedings of the IEEE Conference on Decision & Control, including the Symposium on Adaptive Processes, pp. 4046–4052

Abstract

Changing conditions or environments can cause system dynamics to vary over time. To ensure optimal control performance, controllers should adapt to these changes. When the underlying cause and time of change is unknown, we need to rely on online data for this adaptation. In this paper, we will use time-varying Bayesian optimization (TVBO) to tune controllers online in changing environments using appropriate prior knowledge on the control objective and its changes. Two properties are characteristic of many online controller tuning problems: First, they exhibit incremental and lasting changes in the objective due to changes to the system dynamics, e.g., through wear and tear. Second, the optimization problem is convex in the tuning parameters. Current TVBO methods do not explicitly account for these properties, resulting in poor tuning performance and many unstable controllers through over-exploration of the parameter space. We propose a novel TVBO forgetting strategy using Uncertainty-Injection (UI), which incorporates the assumption of incremental and lasting changes. The control objective is modeled as a spatio-temporal Gaussian process (GP) with UI through a Wiener process in the temporal domain. Further, we explicitly model the convexity assumptions in the spatial dimension through GP models with linear inequality constraints. In numerical experiments, we show that our model outperforms the state-of-the-art method in TVBO, exhibiting reduced regret and fewer unstable parameter configurations.

Authors 3

1. Paul Brunzema Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Institute for Data Science in Mechanical Engineering,Aachen,Germany

   Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany

2. Alexander von Rohr Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University · Max Planck Institute for Intelligent Systems · Ingenieurgesellschaft Auto und Verkehr (Germany)

   Affiliation as printed

   RWTH Aachen University,Institute for Data Science in Mechanical Engineering,Aachen,Germany

   IAV GmbH Ingenieurgesellschaft Auto und Verkehr, Germany

   Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany

   Max Planck Institute for Intelligent Systems, Stuttgart, Germany

3. Sebastian Trimpe Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University · Max Planck Institute for Intelligent Systems

   Affiliation as printed

   RWTH Aachen University,Institute for Data Science in Mechanical Engineering,Aachen,Germany

   Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany

   Max Planck Institute for Intelligent Systems, Stuttgart, Germany