A Theory-Informed Roadmap Towards Multimodal Ocular-based Attention Metrics for Conversational VR

ACM Symposium on Applied Perception, pp. 1–2

Abstract

Eye‑tracking in consumer head-mounted displays (HMDs) already provides reliable data for visual attention tracking. We now strive to extend this capability to conversational attention, which also involves auditory processing, cognitive load, mind‑wandering, and fatigue. To this end, we propose a taxonomy that maps gaze, blink dynamics, and pupillometry onto these five complementary dimensions and briefly summarize current eye‑tracking‑based evaluation methods. From the identified gaps, we outline a three‑stage research roadmap — (i) robust pupillometry calibration, (ii) multimodal signal disambiguation, and (iii) composite attention‑score fusion — aimed at achieving reliable, real‑time inference of attentional engagement in Virtual-Reality (VR)‑based dialogues.

Authors 3

1. Agnes Widera Aachen Visual Computing Institute

   RWTH Aachen University

   Affiliation as printed

   Visual Computing Institute, RWTH Aachen University, Aachen, Germany

2. Torsten Kuhlen Aachen Visual Computing Institute

   RWTH Aachen University

   Affiliation as printed

   Visual Computing Institute, RWTH Aachen University, Aachen, Germany

3. Andrea Bönsch Aachen Visual Computing Institute

   RWTH Aachen University

   Affiliation as printed

   Visual Computing Institute, RWTH Aachen University, Aachen, Germany