Synergizing natural visual features and 3D building models for robust indoor localization in mixed reality environments

Geo-spatial Information Science, vol. 28, pp. 3152–3177

Abstract

The rapidly developing Mixed Reality (MR) technology provides a powerful visualization and interaction approach for outdoor and indoor environments. A main challenge of MR is to accurately align (register) virtual information or objects to the real world, which is a prerequisite for a good visualization and immersive experience. This paper introduces a visual localization method for MR based on the natural feature door and as-planned Building Information Modeling (BIM) models as reference data. The proposed method incorporates a YOLOv5-assisted and AprilTag-inspired 2D door corner detector as well as an interactive extraction approach for 3D door corners from BIM models. A Perspective-n-Point (PnP) method is applied to estimate the pose of the MR device. The proposed method was deployed and evaluated on the Microsoft HoloLens 2. The results indicate that as-planned BIM models and natural visual features such as doors prove to be an effective approach to achieve state-of-the-art localization accuracy in indoor environments.

Authors 2

1. Christoph Blut
2. Jörg Blankenbach Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen University