Saving Energy and Spectrum in Enabling URLLC Services: A Scalable RL Solution

IEEE Transactions on Industrial Informatics, vol. 19, pp. 10265–10276

Abstract

Communication systems supporting cyber-physical production applications should satisfy stringent delay and reliability requirements. Diversity techniques and power control are the main approaches to reduce latency and enhance the reliability of wireless communications at the expense of redundant transmissions and excessive resource usage. Focusing on the application layer reliability key performance indicators (KPIs), we design a deep reinforcement learning orchestrator for power control and hybrid automatic repeat request retransmissions to optimize these KPIs. Furthermore, to address the scalability issue that emerges in the per-device orchestration problem, we develop a new branching soft actor-critic framework in which a separate branch represents the action space of each industrial device. Our orchestrator enables near real-time control and can be implemented in the edge cloud. We test our solution with a 3GPP-compliant and realistic simulator for factory automation scenarios. Compared to the state-of-the-art, our solution offers significant scalability gains in terms of computational time and memory requirements. Our extensive experiments show significant improvements in our target KPIs, over the state-of-the-art, especially for 5th percentile user availability. To achieve these targets, our framework requires substantially less total energy or spectrum, thanks to our scalable RL solution.

Authors 5

1. Milad Ganjalizadeh Aachen

   RWTH Aachen University · KTH Royal Institute of Technology · Ericsson (Sweden)

   Affiliation as printed

   School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden

   -basi are with Ericsson Research, Ericsson AB, 164 40 Stock-holm, Sweden

   Ericsson AB, 164 40 Stockholm, Sweden

   RWTH Aachen University, 520 62 Aachen, Germany

   School of EECS, KTH Royal Institute of Technology, Stockholm, Sweden and

2. Hossein Shokri‐Ghadikolaei Aachen

   RWTH Aachen University · KTH Royal Institute of Technology · Ericsson (Sweden)

   Affiliation as printed

   Ericsson Research, Ericsson AB, Stockholm, Sweden

   -basi are with Ericsson Research, Ericsson AB, 164 40 Stock-holm, Sweden

   Ericsson AB, 164 40 Stockholm, Sweden

   RWTH Aachen University, 520 62 Aachen, Germany

   School of EECS, KTH Royal Institute of Technology, 114 28 Stockholm, Sweden, and Ericsson Research,

   School of EECS, KTH Royal Institute of Technology, Stockholm, Sweden and

3. Amin Azari Aachen

   RWTH Aachen University · KTH Royal Institute of Technology · Ericsson (Sweden)

   Affiliation as printed

   Ericsson Research, Ericsson AB, Stockholm, Sweden

   -basi are with Ericsson Research, Ericsson AB, 164 40 Stock-holm, Sweden

   Ericsson AB, 164 40 Stockholm, Sweden

   RWTH Aachen University, 520 62 Aachen, Germany

   School of EECS, KTH Royal Institute of Technology, 114 28 Stockholm, Sweden, and Ericsson Research,

   School of EECS, KTH Royal Institute of Technology, Stockholm, Sweden and

4. Abdulrahman Alabbasi Aachen

   RWTH Aachen University · KTH Royal Institute of Technology · Ericsson (Sweden)

   Affiliation as printed

   Ericsson Research, Ericsson AB, Stockholm, Sweden

   -basi are with Ericsson Research, Ericsson AB, 164 40 Stock-holm, Sweden

   Ericsson AB, 164 40 Stockholm, Sweden

   RWTH Aachen University, 520 62 Aachen, Germany

   School of EECS, KTH Royal Institute of Technology, 114 28 Stockholm, Sweden, and Ericsson Research,

   School of EECS, KTH Royal Institute of Technology, Stockholm, Sweden and

5. Marina Petrova Aachen

   RWTH Aachen University · KTH Royal Institute of Technology · Ericsson (Sweden)

   Affiliation as printed

   School of Electrical Engineering and Computer Science, KTH Royal Institute of Technology, Stockholm, Sweden

   -basi are with Ericsson Research, Ericsson AB, 164 40 Stock-holm, Sweden

   Ericsson AB, 164 40 Stockholm, Sweden

   RWTH Aachen University, 520 62 Aachen, Germany

   School of EECS, KTH Royal Institute of Technology, Stockholm, Sweden and