Augmented Reality-Based Drone Pilot Assistance Using Consumer-Grade Hardware

IEEE International Conference on Consumer Electronics - Berlin (ICCE-Berlin), pp. 123–128

Abstract

Unmanned Aerial Systems (UASs), also known as drones, are experiencing a surge in usage across industries and among consumers of diverse backgrounds. This rise in adoption introduces challenges for novice pilots. This paper explores how Augmented Reality (AR) can help address these challenges by providing an intuitive visualization of the drone's pose given the pilot's third-person perspective of the aircraft. We (1) experimentally evaluate if commercially available AR and UAS technology provides the necessary foundation in terms of data and capabilities, (2) propose a digital twin-based assistance system, incorporating a simulation environment, a Microsoft HoloLens 2, and a DJI Mavic Mini drone to enhance the piloting experience and increase safety by visualizing flight information in real time, and (3) implement the system and perform experiments for a first evaluation of the proposed approach.11A commented mixed reality capture video of the final system setup in action is available online at https://youtu.be/R3mcoBQ4NZc.

Authors 3

1. Alexander Atanasyan Aachen Institute for Man-Machine Interaction

   RWTH Aachen University

   Affiliation as printed

   Institute for Man-Machine Interaction, RWTH Aachen University,Aachen,Germany

   Institute for Man-Machine Interaction, RWTH Aachen University, Aachen, Germany

2. Rodrigo Gabriel Cortés Estévez Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University,Aachen,Germany

   RWTH Aachen University, Aachen, Germany

3. Jürgen Roßmann Aachen Institute for Man-Machine Interaction

   RWTH Aachen University

   Affiliation as printed

   Institute for Man-Machine Interaction, RWTH Aachen University,Aachen,Germany

   Institute for Man-Machine Interaction, RWTH Aachen University, Aachen, Germany