Optimized Pulse Patterns Design for Electrical Drives: A Reinforcement Learning Approach

IEEE Open Journal of the Industrial Electronics Society, vol. 7, pp. 905–919

Abstract

Optimal modulation techniques such as optimized pulse patterns (OPPs) are essential for minimizing power losses and hence achieving efficient operation of electrical drive systems. Conventional OPPs are designed offline to minimize different steady-state performance criteria and require switching angles-based analytical models suitable for gradient-based optimization. Such models are however not always available and can be complex and difficult to validate. Hence, a data-driven design approach that can utilize any form of models, or even be used for direct OPP learning, e.g., from a real-world drive, without requiring any modeling effort, is of significant interest. In this paper, we present a first proof of concept for a data-driven design of steady-state OPPs using reinforcement learning (RL). We explore the feasibility of training an end-to-end neural network policy that maps desired working points (WPs) to reference OPPs by interacting with an electrical drive environment. We introduce a novel parametrization of switching angles that inherently incorporates their constraints, and we utilize an efficient simulation set-up to simplify the training for this proof-of-concept investigation. Through extensive simulation and real-world experiments, we demonstrate the potential and trade offs of the proposed RL-based approach for designing loss-optimal pulse patterns for a traction permanent magnet synchronous motor drive. The performance is benchmarked against a conventional, indirectly loss optimized, model-based OPP design. Test bench results show a potential for power loss reduction in learned OPPs compared to conventional ones, despite the simulation model bias, and motivate learning OPPs directly from a real-world drive environment.

Authors 5

1. Mohammad Abu-Ali

   Robert Bosch (Germany)

   Affiliation as printed

   Corporate Research of Robert Bosch GmbH, Renningen, Germany

2. Felix Berkel

   Robert Bosch (Germany)

   Affiliation as printed

   Corporate Research of Robert Bosch GmbH, Renningen, Germany

3. Maximilian Manderla

   Robert Bosch (Germany)

   Affiliation as printed

   Corporate Research of Robert Bosch GmbH, Renningen, Germany

4. Markus Pietschner Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University, Aachen, Germany

5. Daniel Görges

   Rheinland-Pfälzische Technische Universität Kaiserslautern-Landau

   Affiliation as printed

   Department of Electrical and Computer Engineering, RPTU University Kaiserslautern-Landau, Kaiserslautern, Germany