Cloud-Based Reinforcement Learning in Automotive Control Function Development

Vehicles, vol. 5, pp. 914–930

Abstract

Automotive control functions are becoming increasingly complex and their development is becoming more and more elaborate, leading to a strong need for automated solutions within the development process. Here, reinforcement learning offers a significant potential for function development to generate optimized control functions in an automated manner. Despite its successful deployment in a variety of control tasks, there is still a lack of standard tooling solutions for function development based on reinforcement learning in the automotive industry. To address this gap, we present a flexible framework that couples the conventional development process with an open-source reinforcement learning library. It features modular, physical models for relevant vehicle components, a co-simulation with a microscopic traffic simulation to generate realistic scenarios, and enables distributed and parallelized training. We demonstrate the effectiveness of our proposed method in a feasibility study to learn a control function for automated longitudinal control of an electric vehicle in an urban traffic scenario. The evolved control strategy produces a smooth trajectory with energy savings of up to 14%. The results highlight the great potential of reinforcement learning for automated control function development and prove the effectiveness of the proposed framework.

Authors 5

1. Lucas Koch Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University 52074 Aachen, Germany

2. Dennis Roeser
   Affiliation as printed

   dSPACE GmbH, Rathenaustraße 26, 33102 Paderborn, Germany

3. Kevin Badalian Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University 52074 Aachen, Germany

4. Alexander Lieb Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University 52074 Aachen, Germany

5. Jakob Andert corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University 52074 Aachen, Germany