HEVC overview

This paper provides an overview of the intra coding techniques in the High Efficiency Video Coding (HEVC) standard being developed by the Joint Collaborative Team on Video Coding (JCT-VC). The intra coding framework of HEVC follows that of traditional hybrid codecs and is built on spatial sample prediction followed by transform coding and postprocessing steps. Novel features contributing to the increased compression efficiency include a quadtree-based variable block size coding structure, block-size agnostic angular and planar prediction, adaptive pre-and postfiltering, and prediction direction-based transform coefficient scanning. This paper discusses the design principles applied during the development of the new intra coding methods and analyzes the compression performance of the individual tools. Computational complexity of the introduced intra prediction algorithms is analyzed both by deriving operational cycle counts and benchmarking an optimized implementation. Using objective metrics, the bitrate reduction provided by the HEVC intra coding over the H.264/advanced video coding reference is reported to be 22% on average and up to 36%. Significant subjective picture quality improvements are also reported when comparing the resulting pictures at fixed bitrate.

Potential Usage of Some SPS Parameters log2_min_luma_coding_block_size-specify the minimal CU size. Potential usage: if a priory known that a video sequence is "flat" or "smooth" then it's worth to consider setting log2_min_luma_coding_block_size = 4 (16x16). Otherwise split-flags at the depth 16x16 are redundantly signaled. log2_diff_max_min_luma_coding_block_sizetogether with log2_min_luma_coding_block_size specify CTU size. There is no reason (excepting maybe a legacy to H.264/AVC) to set CTU size smaller than 64x64. Moreover, according to [8], 64 × 64-sized CTU brings nearly 12% bitrate reduction on the average compared with 16×16-sized CTU. log2_min_transform_block_sizespecify the minimal transform block size. Potential usage, in case of "flat" video sequence it's worth to consider setting log2_min_transform_block_size to 8x8. log2_diff_max_min_transform_block_size-together with the minimal TB size specifies the maximal TB size. Large transform sizes can cause performance peaks therefore it's worth consider to avoid 32x32 transforms by setting maximal transform size to 16x16. High-Level Syntax (PPS Header) Note: There are two IDR frame types: IDR_W_RADL and IDR_N_LP. The first type of IDR enables presence of leading frames (i.e. RADL frames) while the second one disallows all leading frames associated with current IDR. Decoding order: Presentation order: Leading pictures Relationship Between RASL and CRA Frames Relationship between CRA and RASL frames is specified by the following Note from the HEVC/H.265: NOTE-A CRA picture contains only I slices, and may be the first picture in the bitstream in decoding order, or may appear later in the bitstream. A CRA picture may have associated RADL or RASL pictures. When a CRA picture has NoRaslOutputFlag equal to 1, the associated RASL pictures are not output by the decoder, because they may not be decodable, as they may contain references to pictures that are not present in the bitstream. The flag NoRaslOutputFlag is either 1 or it's specified by vague 'some external means'. Anyway, if you start decoding at a CRA frame in the middle of a stream, you can almost safety discard RASL frames without any risk to incur visual impairments due to lacking reference. Picture Syntax (2) Each picture is divided into coding tree units (CTUs, alternative of MBs in AVC/H.264) with a per-video defined size of 16x16, 32x32 or 64x64 (the size is limited by the level and signaled in SPS). • Each 64x64 CTU can be recursively divided into four smaller CUs until the size 8x8. Coding Tree Block (CTB): The size N of the CTBs is chosen by the encoder (16x16, 32x32, 64x64). Luma CTB covers a square picture area of N ×N samples and the corresponding chroma CTBs cover each (N/2) × (N/2) samples (in 4:2:0 format). Coding Tree Units (CTU): The luma CTB and the two chroma CTBs, together with the associated syntax, form a coding tree unit (CTU). The CTU is the basic processing unit similar to MB in prior standards. Coding Block (CB): Each CTB can be further partitioned into multiple coding blocks (CBs). The size of the CB can range from the same size as the CTB to the minimum size (8×8). Coding Unit (CU) The luma CB and the chroma CBs, together with the associated syntax, form a coding unit (CU). Each CU can be either Intra or Inter predicted. Actually CU is the basic unit for compression. CTU Syntax 64x64 CTU CTU Syntax (2) All CUs in a CTU are encoded (traversed) in Z-Scan (left-to-right and depth-first traversal) order, this order makes top and left samples to be available (casual) in most cases : 64x64 CTU The figure taken: Benjamin Bross: "Relax it's only HEVC", WBU

Journal of Real-Time Image Processing, 2017

The high efficiency video coding (HEVC) is the new video coding standard, which obtains over 50% bit rate savings compared with H.264/AVC for the same perceptual quality. Intra-prediction coding in HEVC achieves high coding performance in expense of high computational complexity, due to the exhaustive evaluation of all available coding units (CU) sizes, with up to 35 prediction modes for each CU, selecting the one with the lower rate distortion cost, among other new features. This paper presents a Unified Architecture to form a novel fast HEVC intra-prediction coding algorithm, denoted as fast partitioning and mode decision. This approach combines a fast partitioning decision algorithm, based on decision trees, which are trained using machine learning techniques, and a fast mode decision algorithm, based on a novel texture orientation detection algorithm, which computes the mean directional variance along a set of co-lines with rational slopes using a sliding window over the prediction unit. Both algorithms proposed apply a similar approach, exploiting the strong correlation between several image features and the optimal CTU partitioning and the optimal prediction mode. The key point of the combined approach is that both algorithms compute the image features with low complexity, and the partition decision and the mode decision can also be taken with low complexity, using decision trees (if-else statements) and by selecting the minimum directional variance between a reduced set of directions. This approach can be implemented using any combination of nodes, obtaining a wide range of time savings, from 44 to 67%, and light penalties from 1.1 to 4.6%. Comparisons with similar state-of-the-art works show the proposed approach achieves the best trade-off between complexity reduction and rate distortion.

IEEE Transactions on Circuits and Systems for Video Technology, 2016

IEEE Transactions on Circuits and Systems for Video Technology

Efficient representation and coding of fine-granular motion information is one of the key research areas for exploiting inter-frame correlation in video coding. Representative techniques towards this direction are affine motion compensation (AMC), decoder-side motion vector refinement (DMVR), and subblock-based temporal motion vector prediction (SbTMVP). Fine-granular motion information is derived at subblock level for all the three coding tools. In addition, the obtained inter prediction can be further refined by two optical flow-based coding tools, the bi-directional optical flow (BDOF) for bi-directional inter prediction and the prediction refinement with optical flow (PROF) exclusively used in combination with AMC. The aforementioned five coding tools have been extensively studied and finally adopted in the Versatile Video Coding (VVC) standard. This paper presents technical details of each tool and highlights the design elements with the consideration of typical hardware implementations. Following the common test conditions defined by Joint Video Experts Team (JVET) for the development of VVC, 5.7 % bitrate reduction on average is achieved by the five tools. For test sequences characterized by large and complex motion, up to 13.4 % bitrate reduction is observed. Additionally, visual quality improvement is demonstrated and analyzed. Index Terms-Versatile video coding (VVC), inter prediction, affine motion compensation (AMC), decoder-side motion vector refinement (DMVR), subblock-based temporal motion vector prediction (SbTMVP), bi-directional optical flow (BDOF), prediction refinement with optical flow (PROF). I. INTRODUCTION V IDEO coding standards play an increasingly important role in diversified video applications and services ranging from the conventional television broadcasting, internet

High Efficiency Video Coding (HEVC) is currently being prepared as the newest video coding standard of the ITU-T Video Coding Experts Group and the ISO/IEC Moving Picture Experts Group. The main goal of the HEVC standardization effort is to enable significantly improved compression performance relative to existing standards in the range of 50% bit-rate reduction for equal perceptual video quality. Intra-frame coding is essential in both still image and video coding. In the block-based coding scheme, the spatial redundancy can be removed by utilizing the correlation between the current pixel and its neighboring reconstructed pixels from the differential pulse code modulation (DPCM) in the early video coding standard H.261 to the angular intra prediction in the latest H.265/HEVC [1], different intra prediction schemes are employed. Almost without exception, linear filters are used in these prediction schemes. This paper provides an overview of the Intra-frame coding techniques of the HEVC standard.

2014 IEEE International Conference on Multimedia and Expo (ICME), 2014

Determining the best partitioning structure for a given Coding Tree Unit (CTU) is one of the most time consuming operations within the HEVC encoder. The brute force search through quadtree hierarchy has a significant impact on the encoding time of high definition (HD) videos. This paper presents a fast coding unit size decision-taking algorithm for intra prediction in HEVC. The proposed algorithm utilizes a low complex texture analysis technique based on the local range property of a pixel in a given neighborhood. Simulation results show that the proposed algorithm achieves an average of 72.24% encoding time efficiency improvement with similar rate distortion performance compared to HEVC reference software HM12.0 for HD videos.

IEEE Signal Processing Letters

A novel intra prediction algorithm is proposed to improve the coding performance of screen content for the emerging Versatile Video Coding (VVC) standard. The algorithm, called In-Loop Residual coding with Scalar Quantization (ILR-SQ), employs in-block pixels as reference rather than the regular out-block ones. To this end, an additional in-loop residual signal is used to partially reconstruct the block at the pixel level, during the prediction. The proposed algorithm is essentially designed to target high detail textures, where deep block partitioning structure is required. Therefore, it is implemented to operate on 4 × 4 blocks only, where further block split is not allowed and the standard algorithm is still unable to properly predict the texture. Experiments in the Joint Exploration Model (JEM) reference software show that the proposed algorithm brings a BD-rate gain of 13% on synthetic content, with a negligible computational complexity overhead at both encoder and decoder sides.

IEEE Signal Processing Letters, 2010

A new intra-prediction mode for the H.264/AVC standard is proposed. Each pixel within a block is predicted by a weighted sum of its neighbours, according to an th order Markov linear model. The weights are obtained through a least-squares estimate from reconstructed data in the neighbouring blocks, so that no overhead is necessary to convey the weights to the decoder. Results show significant improvements in H.264/AVC compression for images that are rich in directional structures, and moderate improvements for the other images and sequences tested.

— Improved video coding techniques introduced in the H.265/High Efficiency Video Coding (HEVC) standard allow video encoders to achieve better compression efficiencies. On the other hand, the increased complexity requires a new design methodology able to face challenges associated with ever higher spatiotemporal resolutions. This paper presents a computationally scalable algorithm and its hardware architecture able to support intra encoding up to 2160p@30 frames/s resolution. The scalability allows a tradeoff between the throughput and the compression efficiency. In particular, the encoder is able to check a variable number of candidate modes. The rate estimation based on bin counting and the distortion estimation in the transform domain simplify the rate–distortion analysis and enable the evaluation of a great number of candidate intra modes. The encoder preselects candidate modes by the processing of 8 × 8 predictions computed from original samples. The preselection shares hardware resources used for the processing of predictions generated from reconstructed samples. To support intra 4×4 modes for the 2160p@30 frames/s resolution, the encoder incorporates a separate reconstruction loop. The processing of blocks with different sizes is interleaved to compensate for the delay of reconstruction loops. Implementation results show that the encoder utilizes 1086k gates and 52-kB on-chip memories for TSMC 90 nm. The main reconstruction loop can operate at 400 MHz, whereas the remaining modules work at 200 MHz. For 2160p@30 frames/s videos, the average BD-rate is 5.46% compared with that of the HM software. Index Terms— Field-programmable gate array (FPGA), H.265/High Efficiency Video Coding (HEVC), intra prediction, video coding, very-large-scale integration (VLSI) architecture.