Action Space-Independent Exploration Methods in Multi-Agent Deep Reinforcement Learning for Wireless Power Allocation

IEEE Wireless Communications and Networking Conference, pp. 1–6

Abstract

Multi-agent deep reinforcement learning has pre-dominantly been applied to tackle the challenges of power allocation within complex and dynamic wireless networks. These intelligent agents rely on exploration as a fundamental tool to gather crucial information about their environment, reducing uncertainty and facilitating more informed power allocation decisions. Importantly, as a wireless channel is inherently dynamic, exploration is an indispensable mechanism for agents to continually adapt by acquiring fresh environment insights. Traditionally, most research has leaned towards employing semi-uniform distributed exploration for discrete action spaces and noise perturbation exploration for continuous action spaces. In this paper, we set out to question the validity of this default choice. We modify both exploration methods to work in both action spaces and with different exploration speeds. We demonstrate that semi-uniform distributed exploration yields comparable results to noise perturbation exploration in a continuous action space while significantly simplifying the exploration factor's fine-tuning. In contrast, noise perturbation exploration excels primarily in continuous action space.

Authors 3

1. Amna Kopic Aachen Chair for Distributed Signal Processing

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair for Distributed Signal Processing,Aachen,Germany

2. Erma Perenda Aachen Chair for Distributed Signal Processing

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair for Distributed Signal Processing,Aachen,Germany

3. Haris Gacanin Aachen Chair for Distributed Signal Processing

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Chair for Distributed Signal Processing,Aachen,Germany