Sim2Real Deep Reinforcement Learning of Compliance-based Robotic Assembly Operations

International Conference on Methods and Models in Automation and Robotics, pp. 300–305

Abstract

Reinforcement learning (RL) enables robots to learn goal-oriented behavior. In production processes with high variances, such as joining operations in end-of-line assembly, this is particularly interesting to save significant programming effort. Due to a large amount of required training data, simulative training is becoming increasingly important. In this paper, we present an approach to learn a contact-rich peg-in-hole assembly task utilizing deep reinforcement learning (DRL) and a compliant robot controller. The DRL-Agent learns directly in the Cartesian space (task space) and not in the joint space of the robot, to increase the robustness and efficiency of the algorithms. To further increase the robustness of the policy and to shorten training times, geometric limitations are imposed by introducing an admissible workspace using a trajectory generator. Furthermore, these limitations result in nearly identical behavior in the simulation and on the real robot, allowing the DRL training process to be purely simulative. The learned policy is experimentally investigated both in the simulation environment and on a real robot, to evaluate its transferability from simulation to reality (sim2real).

Authors 5

1. Oliver Petrović Aachen Laboratory for Machine Tools and Production Engineering (WZL) Department of Automation and Control Chair of Machine Tools

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Laboratory for Machine Tools and Production Engineering (WZL),Department of Automation and Control Chair of Machine Tools,Aachen,Germany,52074

2. Lukas Schäper Aachen Laboratory for Machine Tools and Production Engineering (WZL) Department of Automation and Control Chair of Machine Tools

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Laboratory for Machine Tools and Production Engineering (WZL),Department of Automation and Control Chair of Machine Tools,Aachen,Germany,52074

3. Simon Roggendorf Aachen Laboratory for Machine Tools and Production Engineering (WZL) Department of Automation and Control Chair of Machine Tools

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Laboratory for Machine Tools and Production Engineering (WZL),Department of Automation and Control Chair of Machine Tools,Aachen,Germany,52074

4. Simon Storms Aachen Laboratory for Machine Tools and Production Engineering (WZL) Department of Automation and Control Chair of Machine Tools

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Laboratory for Machine Tools and Production Engineering (WZL),Department of Automation and Control Chair of Machine Tools,Aachen,Germany,52074

5. Christian Brecher Aachen Laboratory for Machine Tools and Production Engineering (WZL) Department of Automation and Control Chair of Machine Tools

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Laboratory for Machine Tools and Production Engineering (WZL),Department of Automation and Control Chair of Machine Tools,Aachen,Germany,52074