Sheet metal localization using deep learning and synthetic data

Journal of Intelligent Manufacturing, vol. 37, pp. 399–415

Abstract

Abstract Improving the accuracy of sheet metal localization in industrial machines is of great interest to many automated manufacturing systems. Current vision-based systems typically rely on traditional image processing algorithms to locate the position of sheets in images. However, these algorithms often do not generalize robustly in real production setups. To achieve this, we propose a novel framework consisting of two deep learning models that locate sheets based on their corners, and a data generation pipeline capable of creating the annotated data required to train the models. Evaluation of this framework on real production data shows that the proposed approach locates sheet metal corners highly accurate with an average error of 2.17 pixels, which is at the edge of the theoretically achievable limit defined by the human annotation error in the test dataset. Extensive experiments show that the proposed framework generalizes well and can therefore be used as a backbone for various automated systems for which sheet metal localization is a relevant task.

Authors 5

1. Hannes Behnen corresponding Aachen Intelligence in Quality Sensing Laboratory for Machine Tools and Production Engineering (WZL)

   TRUMPF (Germany) · RWTH Aachen University

   Affiliation as printed

   Intelligence in Quality Sensing, Laboratory for Machine Tools and Production Engineering, WZL, RWTH Aachen University, Campus-Boulevard 30, 52074, Aachen, Germany

   Research and Development, TRUMPF Werkzeugmaschinen SE + Co. KG, Johann-Maus-Str. 2, 71254, Ditzingen, Germany

2. Guillem Boada-Gardenyes corresponding

   TRUMPF (Germany)

   Affiliation as printed

   Research and Development, TRUMPF Werkzeugmaschinen SE + Co. KG, Johann-Maus-Str. 2, 71254, Ditzingen, Germany

3. Robert Heinrich Schmitt Aachen Fraunhofer Institute for Production Technology IPT Intelligence in Quality Sensing Laboratory for Machine Tools and Production Engineering (WZL)

   Fraunhofer Institute for Production Technology IPT · RWTH Aachen University

   Affiliation as printed

   Fraunhofer Institute for Production Technology IPT, Steinbachstr. 17, 52074, Aachen, Germany

   Intelligence in Quality Sensing, Laboratory for Machine Tools and Production Engineering, WZL, RWTH Aachen University, Campus-Boulevard 30, 52074, Aachen, Germany

4. Marc Teschner

   TRUMPF (Germany)

   Affiliation as printed

   Research and Development, TRUMPF Werkzeugmaschinen SE + Co. KG, Johann-Maus-Str. 2, 71254, Ditzingen, Germany

5. Willi Pönitz

   TRUMPF (Germany)

   Affiliation as printed

   Research and Development, TRUMPF Werkzeugmaschinen SE + Co. KG, Johann-Maus-Str. 2, 71254, Ditzingen, Germany