3D Geometry-Based Global Motion Compensation For VVC

Proceedings - International Conference on Image Processing, pp. 2054–2058

Abstract

Since most 2D videos are initially captured in 3D environments, developing 3D motion models for video coding is beneficial. This paper introduces a method for extracting 3D geometry data from 2D videos and synthesizing 3D-based virtual Reference Pictures (RPs). These novel RPs are offered to the Versatile Video Coding (VVC) encoder for use in motion compensation. The proposed method generates 3D geometry data in the form of 3D meshes at the encoder and transmits it to the decoder as an overhead bitstream. However, this overhead could eat up the whole coding gain or even make it worse than the anchor VVC. This paper solves this problem by employing 3D mesh processing techniques, e.g., mesh de-noising, mesh decimation, and mesh compression. Simulation results show that the proposed method outperforms VVC up to ~ 3.8%.

Authors 4

1. Hossein Bakhshi Golestani Aachen Institute of Communications Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute of Communications Engineering, RWTH Aachen University,Germany

   Institute of Communications Engineering, RWTH Aachen University, Germany

2. Johannes Sauer Aachen Institute of Communications Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute of Communications Engineering, RWTH Aachen University,Germany

   Institute of Communications Engineering, RWTH Aachen University, Germany

3. Christian Rohlfmg Aachen Institute of Communications Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute of Communications Engineering, RWTH Aachen University,Germany

   Institute of Communications Engineering, RWTH Aachen University, Germany

4. Jens-Rainer Ohm Aachen Institute of Communications Engineering

   RWTH Aachen University

   Affiliation as printed

   Institute of Communications Engineering, RWTH Aachen University,Germany

   Institute of Communications Engineering, RWTH Aachen University, Germany