Adaptive Control Strategies for Networked Systems: A Reinforcement Learning-Based Approach

Electronics, vol. 14, pp. 1312

Abstract

Advances in industrial 5G communication technologies and robotics create new possibilities while also increasing the complexity and variability of networked control systems. The additional throughput and lower latency provided by 5G networks enable applications such as teleoperation of machinery, flexible reconfigurable robotic manufacturing cells, or automated guided vehicles. These use cases are set up in dynamic network environments where communication latency and jitter become critical factors that must be managed. Despite the advancements in 5G technologies, such as ultra-reliable low-latency communication (URLLC), adaptive control strategies such as reinforcement learning (RL) remain critical to handle unpredictable network conditions and ensure optimal system performance in real-world industrial applications. In this paper, we investigate the potential of RL in scenarios with communication latency similar to a public 5G deployment. Our study includes an incremental improvement by utilizing long short-term memory-based neural networks in combination with proximal policy optimization in this scenario. Our findings indicate that incorporating latency into the training environment enhances the robustness and efficiency of RL controllers, especially in scenarios characterized by variable network delays. This exploration provides insights into the feasibility of using RL for networked control systems and underscores the importance of incorporating realistic network conditions into the training phase.

Authors 4

1. André Gilerson corresponding Aachen Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, 52074 Aachen, Germany

2. Niklas Bünte Aachen Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, 52074 Aachen, Germany

3. Pierre Kehl Aachen Fraunhofer Institute for Production Technology IPT

   Fraunhofer Institute for Production Technology IPT

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, 52074 Aachen, Germany

4. Robert Heinrich Schmitt Aachen Fraunhofer Institute for Production Technology IPT Laboratory for Machine Tools and Production Engineering (WZL) Department Intelligence in Quality Sensing

   Fraunhofer Institute for Production Technology IPT · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, 52074 Aachen, Germany

   Laboratory for Machine Tools and Production Engineering (WZL), Department Intelligence in Quality Sensing, RWTH Aachen University, Campus-Boulevard 30, 52074 Aachen, Germany