A Modular End-to-End Sim2Real Pipeline for Object Detection and 6D-Pose Estimation in Robot-based Industrial Assembly

Procedia CIRP, vol. 146, pp. 780–785

Abstract

Robust perception is a key enabler for flexible robotic assembly. However, the limited availability of training data remains a major obstacle to developing machine learning-based computer vision models for industrial applications. Synthetic data generation offers a promising solution, yet, the Sim2Real gap still causes performance degradation when models are transferred from simulation to reality. Moreover, workflows for synthetic data generation are often time-consuming and require extensive manual effort as well as deep technical expertise. This paper presents a framework for a modular, end-to-end Sim2Real pipeline for object detection and 6D-pose estimation in robot-based industrial assembly, that combines synthetic data generation and machine learning processes in a configurable, adaptable manner. The framework comprises four separate modules for model preparation, data generation, data preparation and modeling, coordinated by an orchestration layer. It enables integration of engineering and domain information for structured domain randomization and adaptation to specific use cases while supporting parameterized experiments for systematic Sim2Real transfer optimization. Industry-related real-world evaluations demonstrate the Sim2Real capabilities of the proposed pipeline framework, achieving a maximum mAP 50 of 0.987 in an object detection task for assembly process monitoring, as well as high accuracy (mean translational error = 3.54 mm, mean rotational error = 1.37°) in a 6D-pose estimation task for robot-based manipulation. When integrated into a physical assembly cell, the trained computer vision model enabled grasping and assembly success rates above 90%, demonstrating that the framework allows efficient perception for robot-based industrial assembly.

Authors 4

1. Oliver Petrović corresponding Aachen Laboratory for Machine Tools WZL

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools WZL of the RWTH Aachen University, Steinbachstr. 19, 52074 Aachen

2. Patrick Dohmen Aachen Laboratory for Machine Tools WZL

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools WZL of the RWTH Aachen University, Steinbachstr. 19, 52074 Aachen

3. Petar Tesic Aachen Laboratory for Machine Tools WZL

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools WZL of the RWTH Aachen University, Steinbachstr. 19, 52074 Aachen

4. Werner Herfs Aachen Laboratory for Machine Tools WZL

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools WZL of the RWTH Aachen University, Steinbachstr. 19, 52074 Aachen