Multi agent reinforcement learning for online layout planning and scheduling in flexible assembly systems

Journal of Intelligent Manufacturing, vol. 35, pp. 3917–3936

Abstract

Abstract Manufacturing systems are undergoing systematic change facing the trade-off between the customer's needs and the economic and ecological pressure. Especially assembly systems must be more flexible due to many product generations or unpredictable material and demand fluctuations. As a solution line-less mobile assembly systems implement flexible job routes through movable multi-purpose resources and flexible transportation systems. Moreover, a completely reactive rearrangeable layout with mobile resources enables reconfigurations without interrupting production. A scheduling that can handle the complexity of dynamic events is necessary to plan job routes and control transportation in such an assembly system. Conventional approaches for this control task require exponentially rising computational capacities with increasing problem sizes. Therefore, the contribution of this work is an algorithm to dynamically solve the integrated problem of layout optimization and scheduling in line-less mobile assembly systems. The proposed multi agent deep reinforcement learning algorithm uses proximal policy optimization and consists of a decoder and encoder, allowing for various-sized system state descriptions. A simulation study shows that the proposed algorithm performs better in 78% of the scenarios compared to a random agent regarding the makespan optimization objective. This allows for adaptive optimization of line-less mobile assembly systems that can face global challenges.

Authors 4

1. Lea Kaven corresponding Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Aachen, Germany

2. Philipp Huke Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Aachen, Germany

3. Amon Göppert Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Aachen, Germany

4. Robert Heinrich Schmitt Aachen Fraunhofer Institute for Production Technology IPT Laboratory for Machine Tools and Production Engineering (WZL)

   Fraunhofer Institute for Production Technology IPT · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, Aachen, Germany

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University, Aachen, Germany