Offline Trajectory Optimization for Offline Reinforcement Learning

ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 4002–4013

Abstract

Offline reinforcement learning (RL) aims to learn policies without online explorations. To enlarge the training data, model-based offline RL learns a dynamics model which is utilized as a virtual environment to generate simulation data and enhance policy learning. However, existing data augmentation methods for offline RL suffer from (i) trivial improvement from short-horizon simulation; and (ii) the lack of evaluation and correction for generated data, leading to low-qualified augmentation.

Authors 7

1. Ziqi Zhao

   Shandong University

   Affiliation as printed

   Shandong University, Qingdao, China

2. Zhaochun Ren Aachen

   Leiden University

   Affiliation as printed

   Leiden University, Leiden, Netherlands

3. Liu Yang

   Shandong University

   Affiliation as printed

   Shandong University, Qingdao, China

4. Fajie Yuan

   Westlake University

   Affiliation as printed

   Westlake University, Hangzhou, China

5. Pengjie Ren

   Shandong University

   Affiliation as printed

   Shandong University, Qingdao, China

6. Zhumin Chen

   Shandong University

   Affiliation as printed

   Shandong University, Qingdao, China

7. Xin Xin

   Shandong University

   Affiliation as printed

   Shandong University, Qingdao, China