Reference picture padding methods for hybrid video coding

RWTH Publications (RWTH Aachen)

Abstract

A common prediction mode in video coding standards is inter prediction. It uses information from an already decoded picture to predict the current one. This is typically achieved by block-based motion compensation from one picture to the next. In this scenario, the source picture is called a reference picture. Boundary handling allows the motion to reach over the border of the reference picture by defining the samples of the outside area. An established form of boundary handling is a nearest neighbor substitution, which can be realized by padding the picture with its outermost sample values. Therefore, the picture boundary handling is also referred to as reference picture padding. A big issue with established boundary handling is that it oftentimes leads to suboptimal predictions, which results in lowered compression efficiency and artifacts at the boundary. Especially when subpictures are introduced, which need boundary handling at their borders, these issues become more severe and artifacts occur not only at the picture borders but also at subpicture borders within the picture. The goal of this thesis is to explore novel solutions for boundary handling to diminish the mentioned issues. However, such solutions may introduce significant additional computational complexity at both the encoder and decoder side. Especially, at the decoder this can be problematic since real-time decoding is a requirement. The goal of the thesis has been to develop a solution that solves these issues with minimum added computational complexity, which can be proposed for adoption into the exploration model of the next Joint Video Experts Team (JVET) standard. To achieve this goal, multiple approaches have been tested and compared with each other to identify those, which result in the highest compression efficiency. Several novel approaches have been developed in the scope of this thesis. A block-based angular extension picture boundary padding algorithm with a signaled angle has been introduced. A novel global angular extension reference picture padding algorithm has been presented. Further, a template matching-based global reference picture padding algorithm has been developed. Since the template matching-based algorithm was identified as the most promising, significant efforts have been made to reduce the added computational complexity to a minimum. Those include an adaptive candidate selection, improved chroma handling, reusing of search results, balancing measures for computational complexity and an early stopping measure to reduce computational complexity. The measures can be combined and form a reference picture padding method that challenges the current state-of-the-art. This approach has been presented to JVET and was accepted into the Enhanced Compression Model (ECM). The solution is especially useful for low-delay applications and for boundary handling of subpictures, which is becoming more and more relevant.

Authors 1

1. Nicolas Neumann corresponding Aachen

   RWTH Aachen University

   Affiliation as printed

   RWTH Aachen