Human motor augmentation with an extra robotic arm without functional interference

Science Robotics, vol. 8, pp. eadh1438

Abstract

Extra robotic arms (XRAs) are gaining interest in neuroscience and robotics, offering potential tools for daily activities. However, this compelling opportunity poses new challenges for sensorimotor control strategies and human-machine interfaces (HMIs). A key unsolved challenge is allowing users to proficiently control XRAs without hindering their existing functions. To address this, we propose a pipeline to identify suitable HMIs given a defined task to accomplish with the XRA. Following such a scheme, we assessed a multimodal motor HMI based on gaze detection and diaphragmatic respiration in a purposely designed modular neurorobotic platform integrating virtual reality and a bilateral upper limb exoskeleton. Our results show that the proposed HMI does not interfere with speaking or visual exploration and that it can be used to control an extra virtual arm independently from the biological ones or in coordination with them. Participants showed significant improvements in performance with daily training and retention of learning, with no further improvements when artificial haptic feedback was provided. As a final proof of concept, naïve and experienced participants used a simplified version of the HMI to control a wearable XRA. Our analysis indicates how the presented HMI can be effectively used to control XRAs. The observation that experienced users achieved a success rate 22.2% higher than that of naïve users, combined with the result that naïve users showed average success rates of 74% when they first engaged with the system, endorses the viability of both the virtual reality-based testing and training and the proposed pipeline.

Authors 10

1. Giulia Dominijanni

   École Polytechnique Fédérale de Lausanne

   Affiliation as printed

   Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

2. Daniel J. L. L. Pinheiro corresponding

   École Polytechnique Fédérale de Lausanne · Universidade Federal de São Paulo

   Affiliation as printed

   Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

   Neuroengineering and Neurocognition Laboratory, Escola Paulista de Medicina, Department of Neurology and Neurosurgery, Division of Neuroscience, Universidade Federal de São Paulo, São Paulo, Brazil

3. Leonardo Pollina corresponding

   École Polytechnique Fédérale de Lausanne

   Affiliation as printed

   Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

4. Bastien Orset

   École Polytechnique Fédérale de Lausanne

   Affiliation as printed

   Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

5. Martina Gini Aachen Neuroelectronic Interfaces Faculty of Electrical Engineering and IT

   Scuola Superiore Sant'Anna · RWTH Aachen University

   Affiliation as printed

   BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant’Anna, Pisa, Italy

   Neuroelectronic Interfaces, Faculty of Electrical Engineering and IT, Rheinisch-Westfälische Technische Hochschule (RWTH) Aachen, Aachen 52074, Germany

   BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy

6. Eugenio Anselmino

   Scuola Superiore Sant'Anna

   Affiliation as printed

   BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant’Anna, Pisa, Italy

   BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy

7. Camilla Pierella

   University of Genoa

   Affiliation as printed

   Department of Neurosciences, Rehabilitation, Ophthalmology, Genetics, and Maternal and Children’s Sciences (DINOGMI), University of Genoa, Genoa, Italy

   Department of Neurosciences, Rehabilitation, Ophthalmology, Genetics, and Maternal and Children's Sciences (DINOGMI), University of Genoa, Genoa, Italy

8. Jérémy Olivier

   HES-SO University of Applied Sciences and Arts Western Switzerland · HES-SO Genève

   Affiliation as printed

   Institute for Industrial Sciences and Technologies, Haute Ecole du Paysage, d’Ingénierie et d’Architecture (HEPIA), HES-SO University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland

9. Solaiman Shokur corresponding

   Scuola Superiore Sant'Anna · École Polytechnique Fédérale de Lausanne

   Affiliation as printed

   BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant’Anna, Pisa, Italy

   Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

   BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy

10. Silvestro Micera corresponding

    Scuola Superiore Sant'Anna · École Polytechnique Fédérale de Lausanne

    Affiliation as printed

    BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant’Anna, Pisa, Italy

    Neuro-X Institute, Ecole Polytechnique Fédérale de Lausanne, Lausanne, Switzerland

    BioRobotics Institute, Health Interdisciplinary Center, and Department of Excellence in AI and Robotics, Scuola Superiore Sant'Anna, Pisa, Italy