Curiosity Driven Reinforcement Learning for Job Shop Scheduling

International Conference on Agents and Artificial Intelligence (ICAART), pp. 216–227

Abstract

The Job Shop Problem (JSP) is a well-known NP-hard problem with numerous applications in manufacturing and other fields.Efficient scheduling is critical for producing customized products in the manufacturing industry in time.Typically, the quality metrics of a schedule, such as the makespan, can only be assessed after all tasks have been assigned, leading to sparse reward signals when framing JSP as a reinforcement learning (RL) problem.Sparse rewards pose significant challenges for many RL algorithms, often resulting in slow learning behavior.Curiosity algorithms, which introduce intrinsic reward signals, have been shown to accelerate learning in environments with sparse rewards.In this study, we explored the effectiveness of the Intrinsic Curiosity Module (ICM) and Episodic Curiosity (EC) by benchmarking them against state-of-the-art methods.Our experiments demonstrate that the use of curiosity significantly increases the amount of states encountered by the RL agent.When the intrinsic and extrinsic reward signals are of comparable magnitude, the agent is with ICM module are able to escape local optima and discover better solutions.

Authors 5

1. Alexander Nasuta Aachen Chair of Intelligence in Quality Sensing

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligence in Quality Sensing, RWTH Aachen, Aachen, Germany

2. Marco Kemmerling Aachen Chair of Intelligence in Quality Sensing

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligence in Quality Sensing, RWTH Aachen, Aachen, Germany

3. Hans Aoyang Zhou Aachen Chair of Intelligence in Quality Sensing

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligence in Quality Sensing, RWTH Aachen, Aachen, Germany

4. Anas Abdelrazeq Aachen Chair of Intelligence in Quality Sensing

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligence in Quality Sensing, RWTH Aachen, Aachen, Germany

5. Robert Heinrich Schmitt Aachen Chair of Intelligence in Quality Sensing

   RWTH Aachen University

   Affiliation as printed

   Chair of Intelligence in Quality Sensing, RWTH Aachen, Aachen, Germany