Automating Manual Tasks through Intuitive Robot Programming and Cognitive Robotics

arXiv (Cornell University)

Abstract

This paper presents a novel concept for intuitive end-user programming of robots, inspired by natural interaction between humans. Natural language and supportive gestures are translated into robot programs using large language models (LLMs) and computer vision (CV). Through equally natural system feedback in the form of clarification questions and visual representations, the generated program can be reviewed and adjusted, thereby ensuring safety, transparency, and user acceptance.

Authors 4

1. Bijan Kavousian Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University , Steinbachstraße 19 , D-52074 Aachen

2. Petar Tesic Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University , Steinbachstraße 19 , D-52074 Aachen

3. Oliver Petrović Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University , Steinbachstraße 19 , D-52074 Aachen

4. C. Brecher Aachen Laboratory for Machine Tools and Production Engineering (WZL)

   RWTH Aachen University

   Affiliation as printed

   Laboratory for Machine Tools and Production Engineering (WZL) of RWTH Aachen University , Steinbachstraße 19 , D-52074 Aachen