Simulation-as-a-Service for Reinforcement Learning Applications by Example of Heavy Plate Rolling Processes

Procedia Manufacturing, vol. 51, pp. 897–903

Abstract

In the production industry, the digital transformation enables a significant optimization potential. The concept of reinforcement learning offers a suitable approach to train agents on learning control strategies, further advancing automation. While applications training directly on real-world processes are rare due to economical and safety constraints, simulations offer a way to develop and evaluate agents prior to deployment. With the rise of service-based business models, the simulation owner and the machine learning expert are likely to be different stakeholders in a joint project. Due to different requirements for both simulations and reinforcement-learning agents, the stakeholders may be reluctant or unable to grant full access to the respective software. This poses a serious impediment to the potential of the digital transformation. In this paper, a distributed architecture is proposed, which allows the remote training of reinforcement learning agents on a simulation. It is shown that this architecture allows the cooperation between two stakeholders by exposing a suitable technical interface to the simulation. The proposed architecture is implemented for a simulation of the multi-step metal forming process of heavy plate rolling. Furthermore, the implemented architecture is used to successfully train a reinforcement-learning agent on the task of designing optimal parameter schedules.

Authors 8

1. Christian Scheiderer corresponding Aachen Institute for Information Management in Mechanical Engineering

   University of Wuppertal · RWTH Aachen University

   Affiliation as printed

   Institute for Information Management in Mechanical Engineering, RWTH Aachen University, Aachen, Germany

   Institute of Technologies and Management of the Digital Transformation University of Wuppertal, Wuppertal, Germany

2. Timo Thun Aachen Institute for Information Management in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Information Management in Mechanical Engineering, RWTH Aachen University, Aachen, Germany

3. Christian Idzik Aachen Institute of Metal Forming

   RWTH Aachen University

   Affiliation as printed

   Institute of Metal Forming, RWTH Aachen University, Aachen, Germany

4. Andrés Felipe Posada-Moreno Aachen Institute for Information Management in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Information Management in Mechanical Engineering, RWTH Aachen University, Aachen, Germany

5. Alexander Krämer Aachen Institute of Metal Forming

   RWTH Aachen University

   Affiliation as printed

   Institute of Metal Forming, RWTH Aachen University, Aachen, Germany

6. Johannes Lohmar Aachen Institute of Metal Forming

   RWTH Aachen University

   Affiliation as printed

   Institute of Metal Forming, RWTH Aachen University, Aachen, Germany

7. Gerhard Hirt Aachen Institute of Metal Forming

   RWTH Aachen University

   Affiliation as printed

   Institute of Metal Forming, RWTH Aachen University, Aachen, Germany

8. Tobias Meisen

   University of Wuppertal

   Affiliation as printed

   Institute of Technologies and Management of the Digital Transformation University of Wuppertal, Wuppertal, Germany