Safe Reinforcement Learning-Based Control for Hydrogen Diesel Dual-Fuel Engines

IFAC-PapersOnLine, vol. 59, pp. 19–24

Abstract

The urgent energy transition requirements towards a sustainable future span multiple industries and are a significant challenge facing humanity. Hydrogen promises a clean, carbon-free future, with the potential to integrate into existing transportation technologies. However, adding hydrogen to existing technologies such as diesel engines requires additional modeling effort. Reinforcement Learning (RL) enables interactive data-driven learning that eliminates the need for mathematical modeling for controller synthesis. The algorithms, however, may not be real-time capable and need large amounts of data to work in practice. This paper presents a novel approach which uses offline model learning with RL to demonstrate safe control of a 4.5 L Hydrogen Diesel Dual-Fuel (H2DF) engine. An offline H2DF model learning step facilitates the policy search in a simulated environment. The controllers are demonstrated to be constraint-compliant and can leverage a novel state-augmentation approach for sample-efficient learning. The offline policy is subsequently experimentally validated on the real engine where the control algorithm is executed on a Raspberry Pi controller and requires 6 times less computation time compared to online model predictive control optimization.

Authors 5

1. Alexander Winkler Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University, Aachen, Germany

2. Armin Norouzi

   University of Alberta

   Affiliation as printed

   Department of Mechanical Engineering, University of Alberta, Edmonton, Canada

3. Hongsheng Guo

   National Research Council Canada

   Affiliation as printed

   National Research Council, Ottawa, Canada

4. Jakob Andert Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University, Aachen, Germany

5. David C. Gordon

   University of Alberta

   Affiliation as printed

   Department of Mechanical Engineering, University of Alberta, Edmonton, Canada