Reinforcement Learning Based Pushing and Grasping Objects from Ungraspable Poses

Proceedings - IEEE International Conference on Robotics and Automation/Proceedings, pp. 3860–3866

Abstract

Grasping an object when it is in an ungraspable pose is a challenging task, such as books or other large flat objects placed horizontally on a table. Inspired by human manipulation, we address this problem by pushing the object to the edge of the table and then grasping it from the hanging part. In this paper, we develop a model-free Deep Reinforcement Learning framework to synergize pushing and grasping actions. We first pre-train a Variational Autoencoder to extract high-dimensional features of input scenario images. One Proximal Policy Optimization algorithm with the common reward and sharing layers of Actor-Critic is employed to learn both pushing and grasping actions with high data efficiency. Experiments show that our one network policy can converge 2.5 times faster than the policy using two parallel networks. Moreover, the experiments on unseen objects show that our policy can generalize to the challenging case of objects with curved surfaces and off-center irregularly shaped objects. Lastly, our policy can be transferred to a real robot without fine-tuning by using CycleGAN for domain adaption and outperforms the push-to-wall baseline.

Authors 5

1. Hongzhuo Liang

   Universität Hamburg

   Affiliation as printed

   Universität Hamburg,Group TAMS,Dept. of Informatics

2. Cong Lin

   Universität Hamburg

   Affiliation as printed

   Universität Hamburg,Group TAMS,Dept. of Informatics

3. Jianzhi Lyu

   Universität Hamburg

   Affiliation as printed

   Universität Hamburg,Group TAMS,Dept. of Informatics

4. Pingfa Feng

   Tsinghua Shenzhen International Graduate School · Tsinghua University

   Affiliation as printed

   Shenzhen International Graduate School, Tsinghua University,Division of Advanced Manufacturing

   Division of Advanced Manufacturing, Shenzhen International Graduate School, Tsinghua University

5. Jianwei Zhang

   Universität Hamburg

   Affiliation as printed

   Universität Hamburg,Group TAMS,Dept. of Informatics