EVO-RL: Evolutionary-Driven Reinforcement Learning

arXiv (Cornell University)

Abstract

In this work, we propose a novel approach for reinforcement learning driven by evolutionary computation. Our algorithm, dubbed as Evolutionary-Driven Reinforcement Learning (evo-RL), embeds the reinforcement learning algorithm in an evolutionary cycle, where we distinctly differentiate between purely evolvable (instinctive) behaviour versus purely learnable behaviour. Furthermore, we propose that this distinction is decided by the evolutionary process, thus allowing evo-RL to be adaptive to different environments. In addition, evo-RL facilitates learning on environments with rewardless states, which makes it more suited for real-world problems with incomplete information. To show that evo-RL leads to state-of-the-art performance, we present the performance of different state-of-the-art reinforcement learning algorithms when operating within evo-RL and compare it with the case when these same algorithms are executed independently. Results show that reinforcement learning algorithms embedded within our evo-RL approach significantly outperform the stand-alone versions of the same RL algorithms on OpenAI Gym control problems with rewardless states constrained by the same computational budget.

Authors 9

1. Ahmed Hallawa Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University

2. Thorsten Born Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen, University, Aachen, Germany

3. Anke Schmeink Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen, University, Aachen, Germany

4. Guido Dartmann

   Trier University of Applied Sciences

   Affiliation as printed

   Trier University of Applied Sciences, Trier, Germany

5. Arne Peine Aachen

   Universitätsklinikum Aachen

   Affiliation as printed

   University Hospital Aachen, Aachen, Germany

6. Lukas Märtin Aachen

   Universitätsklinikum Aachen

   Affiliation as printed

   University Hospital Aachen, Aachen, Germany

7. Giovanni Iacca

   University of Trento

   Affiliation as printed

   University of Trento, trento, Italy

8. Agoston Endre Eiben

   Vrije Universiteit Amsterdam

   Affiliation as printed

   Vrije Universiteit Amsterdam; Amsterdam The Netherlands

9. Gerd Ascheid Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen, University, Aachen, Germany