Multi-Agent Reinforcement Learning Aided Resources Allocation Method in Vehicular Networks

IEEE Vehicular Technology Conference, pp. 1–5

Abstract

To address the problem of spectrum resources and transmitting power for vehicular networks, this paper proposes a resource allocation (RA) method based on dueling double deep-Q network (D3QN) reinforcement learning (RL). Due to the high mobility of the vehicle, the channel changes rapidly which makes it difficult to accurately collect high-accuracy channel state information at the base station and to perform centralized management. In response of this difficulty, we construct a multi-intelligence model, using Manhattan Grid Layout City Model as the basis of environment and with each vehicle-to-vehicle (V2V) link as an intelligence. They work together to interact with the environment, receive appropriate observations, get rewards, and finally learn to improve the allocation of power and spectrum to enable users to achieve a better entertainment experience and a safer driving environment. Experimental results demonstrate that with proper training mechanism and reward function construction, cooperation among multiple intelligence can be performed in a distributed manner, with improvements in both the capacity of total vehicle-to-infrastructure links and the effective payload delivery success rate of the V2V links compared to common Q-network.

Authors 7

1. Yuxin Ji

   Nanjing University of Posts and Telecommunications

   Affiliation as printed

   College of Telecommunications and Information Engineering, NJUPT,Nanjing,China

   College of Telecommunications and Information Engineering, NJUPT, Nanjing, China

2. Xixi Zhang

   Nanjing University of Posts and Telecommunications

   Affiliation as printed

   College of Telecommunications and Information Engineering, NJUPT,Nanjing,China

   College of Telecommunications and Information Engineering, NJUPT, Nanjing, China

3. Yu Wang

   Nanjing University of Posts and Telecommunications

   Affiliation as printed

   College of Telecommunications and Information Engineering, NJUPT,Nanjing,China

   College of Telecommunications and Information Engineering, NJUPT, Nanjing, China

4. Haris Gacanin Aachen Institute for Communication Technologies and Embedded Systems

   RWTH Aachen University

   Affiliation as printed

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University,Aachen,Germany

   Institute for Communication Technologies and Embedded Systems, RWTH Aachen University, Aachen, Germany

5. Hikmet Sari

   Nanjing University of Posts and Telecommunications

   Affiliation as printed

   College of Telecommunications and Information Engineering, NJUPT,Nanjing,China

   College of Telecommunications and Information Engineering, NJUPT, Nanjing, China

6. Fumiyuki Adachi

   Tohoku University

   Affiliation as printed

   International Research Institute of Disaster Science (IRIDeS), Tohoku University,Sendai,Japan

   International Research Institute of Disaster Science (IRIDeS), Tohoku University, Sendai, Japan

7. Guan Gui

   Nanjing University of Posts and Telecommunications

   Affiliation as printed

   College of Telecommunications and Information Engineering, NJUPT,Nanjing,China

   College of Telecommunications and Information Engineering, NJUPT, Nanjing, China