Adaptive Traffic Light Control With Deep Reinforcement Learning: An Evaluation of Traffic Flow and Energy Consumption

IEEE Transactions on Intelligent Transportation Systems, vol. 24, pp. 15066–15076

Abstract

Cities of all sizes around the world are facing increasing levels of congestion, which leads to increasing travel time and emissions, and ultimately affects the quality of life. Relevant research suggests that adaptive traffic light control systems can improve the traffic flow, but their impact on energy-efficiency of vehicle propulsion systems is not well understood. In this study, we use Proximal Policy Optimization, a Deep Reinforcement Learning algorithm, to develop an optimized adaptive traffic light control systems that controls three traffic lights simultaneously. For this purpose, we have created a microscopic traffic simulation of the city of Aachen, Germany, calibrated on the basis of traffic measurements, where the actual traffic light schedule, a green-wave, fixed-time control scheme, serves as a reference. The traffic simulation is coupled with detailed, physics-based powertrain models of both conventional and electric vehicles, which are validated against chassis dynamometer measurements. By analyzing the complex interactions between traffic light control, the resulting vehicle trajectories and the powertrain components, we show that Reinforcement Learning-based adaptive control can significantly improve the traffic flow, with a 41% increase in average velocity, without any drawbacks in CO2 emission (−1%). Furthermore, we find that maximizing traffic flow and minimizing CO2 emissions are not necessarily contradictory objectives, and identify an increased energy saving potential at low traffic densities. Thus, we prove that adaptive traffic light control can make traffic not only more time-efficient, but also more sustainable.

Authors 5

1. Lucas Koch Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University, Aachen, Germany

2. Tobias Brinkmann Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University, Aachen, Germany

3. Marius Wegener Aachen FEV Europe GmbH

   FEV (Germany)

   Affiliation as printed

   FEV Europe GmbH, Aachen, Germany

4. Kevin Badalian Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University, Aachen, Germany

5. Jakob Andert Aachen

   RWTH Aachen University

   Affiliation as printed

   Teaching and Research Area Mechatronics in Mobile Propulsion, RWTH Aachen University, Aachen, Germany