Behavioral Optimization in a Robotic Serial Reaching Task Using Predictive Information

IEEE Transactions on Cognitive and Developmental Systems, vol. 15, pp. 1012–1019

Abstract

Prediction is a powerful approach to minimize errors and control problems in familiar environments and tasks. In human motor execution of sequential action, context effects can be observed, such as anticipation of or predictive movement toward target objects, where later subactions are affected by the execution of earlier subactions. In this article, we present a simulation framework for a serial reaching task using a 4-DoF robotic arm to examine the learning of context effects in simulated robotic reinforcement learning agents. As we demonstrate, giving robotic agents access to predictive information about a future target object’s identity results in motion optimization, where the identity of the next target modulates earlier subactions. Specifically, agents learn to anticipate and predict the location of the next target object, and move toward it before it appears, thus achieving higher rewards than agents that were not given predictive information.

Authors 3

1. Deniz Sen Aachen Mathematical Institute

   Leiden University

   Affiliation as printed

   Mathematical Institute, Leiden University, Leiden, CA, The Netherlands

2. Roy de Kleijn Aachen Leiden Institute for Brain and Cognition

   Leiden University

   Affiliation as printed

   Leiden Institute for Brain and Cognition, Leiden University, Leiden, CA, The Netherlands

3. George Kachergis

   Stanford University

   Affiliation as printed

   Department of Psychology, Stanford University, Stanford, CA, USA