Reinforcement Learning Agent for Multi-Objective Online Process Parameter Optimization of Manufacturing Processes

Applied Sciences, vol. 15, pp. 7279

Abstract

Optimizing manufacturing processes to reduce scrap and enhance process stability presents significant challenges, particularly when multiple conflicting objectives must be addressed concurrently. As the number of objectives increases, the complexity of the optimization task escalates. This difficulty is further intensified in online optimization scenarios, where optimal parameter settings must be delivered in real time within active production environments. In this work, we propose a reinforcement learning-based framework for the multi-objective optimization of manufacturing parameters, demonstrated through a case study on pinion gear manufacturing. The framework utilizes the Multi-Objective Maximum a Posteriori Optimization (MO-MPO) algorithm to train a reinforcement learning agent. A high-fidelity simulation of the pinion manufacturing process is constructed in Simufact, serving both data generation and validation purposes. The agent’s performance is assessed using a hold-out test set along with additional simulations of the physical process. To ensure the generalizability of the approach, further validation is performed using open-source manufacturing datasets and synthetically generated data. The results demonstrate the feasibility of the proposed method for real-time industrial deployment. Moreover, Pareto-optimality is verified via half-space analysis, emphasizing the framework’s effectiveness in managing trade-offs among competing objectives.

Authors 7

1. Akshay Paranjape corresponding
   Affiliation as printed

   IconPro GmbH, Friedlandstraße 18, 52064 Aachen, Germany

2. Nahid Quader
   Affiliation as printed

   IconPro GmbH, Friedlandstraße 18, 52064 Aachen, Germany

3. Lars Uhlmann Aachen Manufacturing Technology Institute (MTI)

   RWTH Aachen University

   Affiliation as printed

   Manufacturing Technology Institute, MTI of RWTH Aachen University, Campus-Boulevard 30, 52074 Aachen, Germany

4. Benjamin Berkels Aachen Aachen Institute for Advanced Study in Computational Engineering Science (AICES)

   RWTH Aachen University · Aachen Institute for Advanced Study in Computational Engineering Science

   Affiliation as printed

   Aachen Institute for Advanced Study in Computational Engineering Science (AICES), RWTH Aachen, Schinkelstrasse 2, 52062 Aachen, Germany

5. Dominik Wolfschläger Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering, WZL of RWTH Aachen University, Campus-Boulevard 30, 52074 Aachen, Germany

6. Robert Heinrich Schmitt Aachen Laboratory for Machine Tools and Production Engineering (WZL) Fraunhofer Institute for Production Technology IPT

   RWTH Aachen University · Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, Steinbachstr. 17, 52074 Aachen, Germany

   Laboratory for Machine Tools and Production Engineering, WZL of RWTH Aachen University, Campus-Boulevard 30, 52074 Aachen, Germany

7. Thomas Bergs Aachen Fraunhofer Institute for Production Technology IPT Manufacturing Technology Institute (MTI)

   RWTH Aachen University · Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, Steinbachstr. 17, 52074 Aachen, Germany

   Manufacturing Technology Institute, MTI of RWTH Aachen University, Campus-Boulevard 30, 52074 Aachen, Germany