Robot Cooking—Transferring Observations into a Planning Language: An Automated Approach in the Field of Cooking

Eng—Advances in Engineering, vol. 4, pp. 2514–2524

Abstract

The recognition of human activities from video sequences and their transformation into a machine-readable form is a challenging task, which is the subject of many studies. The goal of this project is to develop an automated method for analyzing, identifying and processing motion capture data into a planning language. This is performed in a cooking scenario by recording the pose of the acting hand. First, predefined side actions are detected in the dataset using classification. The remaining frames are then clustered into main actions. Using this information, the known initial positions and virtual object tracking, a machine-readable planning domain definition language (PDDL) is generated.

Authors 5

1. Florian Menz Aachen Institute of Mechanism Theory

   RWTH Aachen University

   Affiliation as printed

   Institute of Mechanism Theory, Machine Dynamics and Robotics, RWTH Aachen University, 52062 Aachen, Germany

2. Ruben Grunau Aachen Institute of Mechanism Theory

   RWTH Aachen University

   Affiliation as printed

   Institute of Mechanism Theory, Machine Dynamics and Robotics, RWTH Aachen University, 52062 Aachen, Germany

3. Nils Mandischer Aachen Institute of Mechanism Theory

   RWTH Aachen University

   Affiliation as printed

   Institute of Mechanism Theory, Machine Dynamics and Robotics, RWTH Aachen University, 52062 Aachen, Germany

4. Mathias Hüsing Aachen Institute of Mechanism Theory

   RWTH Aachen University

   Affiliation as printed

   Institute of Mechanism Theory, Machine Dynamics and Robotics, RWTH Aachen University, 52062 Aachen, Germany

5. Burkhard J. Corves Aachen Institute of Mechanism Theory

   RWTH Aachen University

   Affiliation as printed

   Institute of Mechanism Theory, Machine Dynamics and Robotics, RWTH Aachen University, 52062 Aachen, Germany