All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning

arXiv (Cornell University)

Abstract

Model-based reinforcement learning (MBRL) infers information about the environment from a learned dynamics model and bears the potential to address open problems such as data efficient and safe learning in robotics. However, inaccuracies of the learned dynamics model are typically exploited by the agent, substantially hampering the capabilities of MBRL methods. We present a framework for dealing with inaccuracies of probabilistic models through targeted handling of uncertainty that effectively mitigates model exploitation. We present recent successes in learning directly on hardware and safe exploration, and discuss future directions for uncertainty-aware MBRL.

Authors 5

1. Bernd Frauenknecht Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Data Science in Mechanical Engineering , RWTH Aachen University , 52062 Aachen ,

2. Devdutt Subhasish Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Data Science in Mechanical Engineering , RWTH Aachen University , 52062 Aachen ,

3. Artur Eisele Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Data Science in Mechanical Engineering , RWTH Aachen University , 52062 Aachen ,

4. Friedrich Solowjow Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Data Science in Mechanical Engineering , RWTH Aachen University , 52062 Aachen ,

5. Sebastian Trimpe Aachen Institute for Data Science in Mechanical Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute for Data Science in Mechanical Engineering , RWTH Aachen University , 52062 Aachen ,