Turbocharger Control for Emission Reduction Based on Deep Reinforcement Learning

IFAC-PapersOnLine, vol. 56, pp. 8266–8271

Abstract

The development of embedded systems is a time-consuming process that relies on the calibration of the control systems by experienced engineers and the availability of prototype vehicles. Modern embedded systems must operate with non-linear functions in a wide range of applications. As a result, even experts are unable to fully comprehend all system interconnections, and optimal performance is rarely achieved. Machine Learning (ML)-based techniques can redefine the conventional approach to decision-making problems in this field. Because Reinforcement Learning (RL) algorithms are self-adaptive, near-optimal solutions can be developed with minimal human intervention, providing significant potential for cost and time savings. In this paper, it is demonstrated how RL can be used to design a control function that competes with and outperforms a state-of-the-art controller, while ensuring the stability of the system. In the Model-in-the-Loop (MiL) framework, a method of accelerated training for the control of the turbocharger is presented that considers both performance and emissions criteria. Data efficiency and training robustness were tested for the Proximal Policy Optimization (PPO) algorithm. Selected policies have been evaluated under transient conditions and are shown to enhance the overall boost pressure accuracy by up to 23%, while reducing cumulative NOx and soot emissions by 4% and 10%, respectively, compared to the reference controller. This work demonstrates another step towards the application of the PPO algorithm in the development of embedded systems for complex control problems. Due to its proven robustness and stability, the method is suitable for transfer to real hardware applications.

Authors 5

1. Mario Picerno corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, Aachen, 52074 Germany

2. Lucas Koch corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, Aachen, 52074 Germany

3. Kevin Badalian corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, Aachen, 52074 Germany

4. Sung-Yong Lee corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, Aachen, 52074 Germany

5. Jakob Andert corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, Aachen, 52074 Germany