• If the amount of delay is not crucial for an application, then reliable transport protocols such as TCP that retransmit undelivered packets can be used to guarantee correct decoding of transmitted data.
• If the amount of delay must be kept low, then either data transmission should be error free (e.g., by using managed networks) or the compressed video stream should be error resilient.

High dynamic range imaging

A set of techniques that allows a greater dynamic range of exposures orvalues (i.e., a wider range of values between light and dark areas) than normaldigital imaging techniques. The intention is to accurately represent the widerange of intensity levels found in examples such as exterior scenes thatinclude light-colored items struck by direct sunlight and areas of deep shadow[7].

Random access period

The period of time between the two closest independently decodable frames(pictures).

RD-point

A point in a two-dimensional rate-distortion space where the values ofbitrate and quality metric are used as x- and y-coordinates, respectively.

Visually lossless compression

A form or manner of lossy compression where the data that are lostafter the file is compressed and decompressed is not detectable to the eye;the compressed data appear identical to the uncompressed data [8].

Wide color gamut

A certain complete color subset (e.g., considered in ITU-R BT.2020 [1]) that supports a wider range of colors (i.e., an extendedrange of colors that can be generated by a specific input or output devicesuch as a video camera, monitor, or printer and can be interpreted by a colormodel) than conventional color gamuts (e.g., considered in ITU-R BT.601 [17] or BT.709 [20]).

AI

All-Intra (each picture is intra-coded)

BD-Rate

Bjontegaard Delta Rate

FIZD

just the First picture is Intra-coded, Zero structural Delay

FPS

Frames per Second

GOP

Group of Picture

GPU

Graphics Processing Unit

HBR

High Bitrate Range

HDR

High Dynamic Range

HRD

Hypothetical Reference Decoder

HEVC

High Efficiency Video Coding

IPTV

Internet Protocol Television

LBR

Low Bitrate Range

MBR

Medium Bitrate Range

MOS

Mean Opinion Score

MS-SSIM

Multi-Scale Structural Similarity quality index

PAM

Picture Access Mode

PSNR

Peak Signal-to-Noise Ratio

QoS

Quality of Service

QP

Quantization Parameter

RA

Random Access

RAP

Random Access Period

RD

Rate-Distortion

SEI

Supplemental Enhancement Information

SIMD

Single Instruction, Multiple Data

SNR

Signal-to-Noise Ratio

UGC

User-Generated Content

VDI

Virtual Desktop Infrastructure

VUI

Video Usability Information

WCG

Wide Color Gamut

• Video is encoded in the cloud by software encoders.
• Source video is split into chunks, each of which is encoded separately, in parallel.
• Closed-GOP encoding with intrapicture intervals of 2-5 seconds (or longer) is used.
• Encoding is perceptually optimized. Perceptual quality is important and should be considered during the codec development.

• High encoder complexity (up to 10x and more) can be tolerated since encoding happens once and in parallel for different segments.
• Decoding complexity should be kept at reasonable levels to enable efficient decoder implementation.
• Support and efficient encoding of a wide range of content types and formats is required:
  • High Dynamic Range (HDR), Wide Color Gamut (WCG), high-resolution (currently, up to 4K), and high-frame-rate content are important use cases; the codec should be able to encode such content efficiently.
  • Improvement of coding efficiency at both lower and higher resolutions is important since low resolutions are used when streaming in low-bandwidth conditions.
  • Improvement on both "easy" and "difficult" content in terms of compression efficiency at the same quality level contributes to the overall bitrate/storage savings.
  • Film grain (and sometimes other types of noise) is often present in movies and similar content; this is usually part of the creative intent.
• Significant improvements in compression efficiency between generations of video standards are desirable since this scenario typically assumes long-term support of legacy video codecs.
• Random access points are inserted frequently (one per 2-5 seconds) to enable switching between resolutions and fast-forward playback.
• The elementary stream should have a model that allows easy parsing and identification of the sample components.
• Middle QP values are normally used in streaming; this is also the range where compression efficiency is important for this scenario.
• Scalability or other forms of supporting multiple quality representations are beneficial if they do not incur significant bitrate overhead and if mandated in the first version.

• unicast (e.g., for video on demand), where delay is not crucial; and
• multicast/broadcast (e.g., for transmitting news) where zapping (i.e., stream changing) delay is important.

• Temporal (frame-rate) scalability; and
• Resolution and quality (SNR) scalability.

• Delay should be kept as low as possible (the preferable and maximum end-to-end delay values should be less than 100 ms [9] and 320 ms [2], respectively);
• Temporal (frame-rate) scalability; and
• Error robustness.

• Random access to pictures for downloaded video data;
• Temporal (frame-rate) scalability; and
• Error robustness.

• Random access to pictures for game broadcasting;
• Temporal (frame-rate) scalability; and
• Error robustness.

• Bit depth: 8 and 10 bits (up to 12 bits for a high profile) per color component.
• Color sampling formats:
  • YCbCr 4:2:0
  • YCbCr 4:4:4, YCbCr 4:2:2, and YCbCr 4:0:0 (preferably in different profile(s))
• For profiles with bit depth of 10 bits per sample or higher, support of high dynamic range and wide color gamut.
• Support of arbitrary resolution according to the level constraints for applications in which a picture can have an arbitrary size (e.g., in screencasting).

• Support of configurations with zero structural delay, also referred to as "low-delay" configurations.
  • Note: End-to-end delay should be no more than 320 ms [2], but it is preferable for its value to be less than 100 ms [9].
• Support of efficient random access point encoding (such as intracoding and resetting of context variables), as well as efficient switching between multiple quality representations.
• Support of configurations with nonzero structural delay (such as out-of-order or multipass encoding) for applications without low-delay requirements, if such configurations provide additional compression efficiency improvements.

• Feasible real-time implementation of both an encoder and a decoder supporting a chosen subset of tools for hardware and software implementation on a wide range of state-of-the-art platforms. The subset of real-time encoder tools should provide meaningful improvement in compression efficiency at reasonable complexity of hardware and software encoder implementations as compared to real-time implementations of state-of-the-art video compression technologies such as HEVC/H.265 and VP9.
• High-complexity software encoder implementations used by offline encoding applications can have a 10x or more complexity increase compared to state-of-the-art video compression technologies such as HEVC/H.265 and VP9.

• Temporal (frame-rate) scalability should be supported.

• Tools that are complementary to the error-protection mechanisms implemented on the transport level should be supported.
• The codec should support mechanisms that facilitate packetization of a bitstream for common network protocols.
• Packetization mechanisms should enable frame-level error recovery by means of retransmission or error concealment.
• The codec should support effective mechanisms for allowing decoding and reconstruction of significant parts of pictures in the event that parts of the picture data are lost in transmission.
• The bitstream specification shall support independently decodable subframe units similar to slices or independent tiles. It shall be possible for the encoder to restrict the bitstream to allow parsing of the bitstream after a packet loss and to communicate it to the decoder.

• Bit depth: up to 16 bits per color component.
• Color sampling formats: RGB 4:4:4.
• Auxiliary channel (e.g., alpha channel) support.

• Resolution and quality (SNR) scalability that provides a low-compression efficiency penalty (increase of up to 5% of BD-rate [13] per layer with reasonable increase of both computational and hardware complexity) can be supported in the main profile of the codec being developed by the NETVC Working Group. Otherwise, a separate profile is needed to support these types of scalability.
• Computational complexity scalability (i.e., computational complexity is decreasing along with degrading picture quality) is desirable.

• High-level multicore parallelism: encoder and decoder operation, especially entropy encoding and decoding, should allow multiple frames or subframe regions (e.g., 1D slices, 2D tiles, or partitions) to be processed concurrently, either independently or with deterministic dependencies that can be efficiently pipelined.
• Low-level instruction-set parallelism: favor algorithms that are SIMD/GPU friendly over inherently serial algorithms

• Low bitrate range (LBR) is the range that contains the four lowest bitrates of the ten specified bitrates (one of the four bitrate values is shared with the neighboring range).
• Medium bitrate range (MBR) is the range that contains the four medium bitrates of the ten specified bitrates (two of the four bitrate values are shared with the neighboring ranges).
• High bitrate range (HBR) is the range that contains the four highest bitrates of the ten specified bitrates (one of the four bitrate values is shared with the neighboring range).

Q'k =   min { abs(Q'i - Qk) },
      i in R

QP'k = argmin { abs(Q'i(QP'i) - Qk(QPk)) },
       i in R

QP'9 QP'8  QP'7 QP'6 QP'5 QP'4 QP'3 QP'2 QP'1 QP'0 <+-----
 ^     ^    ^    ^    ^    ^    ^    ^    ^    ^    | Tested
 |     |    |    |    |    |    |    |    |    |    | codec
Q'0   Q'1  Q'2  Q'3  Q'4  Q'5  Q'6  Q'7  Q'8  Q'9  <+-----
 ^               ^              ^              ^
 |               |              |              |
Q0    Q1    Q2   Q3   Q4   Q5   Q6   Q7   Q8   Q9  <+-----
 ^    ^     ^    ^    ^    ^    ^    ^    ^    ^    | Reference
 |    |     |    |    |    |    |    |    |    |    | codec
QP9  QP8   QP7  QP6  QP5  QP4  QP3  QP2  QP1  QP0  <+-----
+----------------+--------------+--------------+--------->
^                ^              ^              ^     Bitrate
|-------LBR------|              |-----HBR------|
                 ^              ^
                 |------MBR-----|

• 25% if calculated over the whole bitrate range; and
• 15% if calculated for each bitrate subrange (LBR, MBR, HBR).

S = min { S_psnr, S_ms-ssim }

[ref-1]

ITU-R, "Parameter values for ultra-high definition television systems for production and international programme exchange", ITU-R Recommendation BT.2020-2, October 2015,
<https://www.itu.int/rec/R-REC-BT.2020-2-201510-I/en>.

[ref-2]

ITU-T, "Quality of Experience requirements for telepresence services", ITU-T Recommendation G.1091, October 2014,
<https://www.itu.int/rec/T-REC-G.1091/en>.

[ref-3]

ISO, "Information technology -- Advanced image coding and evaluation -- Part 1: Guidelines for image coding system evaluation", ISO/IEC TR 29170-1:2017, October 2017,
<https://www.iso.org/standard/63637.html>.

[ref-4]

ISO, "Information technology -- High efficiency coding and media delivery in heterogeneous environments -- Part 2: High efficiency video coding", ISO/IEC 23008-2:2015, May 2018,
<https://www.iso.org/standard/67660.html>.

[ref-5]

ITU-T, "High efficiency video coding", ITU-T Recommendation H.265, November 2019,
<https://www.itu.int/rec/T-REC-H.265>.

[ref-6]

Fraunhofer Institute for Telecommunications, "High Efficiency Video Coding (HEVC) reference software (HEVC Test Model also known as HM)",
<https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/>.

[ref-7]

Federal Agencies Digital Guidelines Initiative, "Term: High dynamic range imaging",
<http://www.digitizationguidelines.gov/term.php?term=highdynamicrangeimaging>.

[ref-8]

Federal Agencies Digital Guidelines Initiative, "Term: Compression, visually lossless",
<http://www.digitizationguidelines.gov/term.php?term=compressionvisuallylossless>.

[9]

ITU-T, S Wenger, "The case for scalability support in version 1 of Future Video Coding", SG 16 (Study Period 2013) Contribution 988, September 2015,
<https://www.itu.int/md/T13-SG16-C-0988/en>.

[10]

YouTube, "Recommended upload encoding settings",
<https://support.google.com/youtube/answer/1722171?hl=en>.

[11]

H Yu, K McCann, R Cohen, and P Amon, "Requirements for an extension of HEVC for coding of screen content", ISO/IEC JTC 1/SC 29/WG 11 Moving Picture Experts Group MPEG2013/N14174, San Jose, USA, January 2014,
<https://mpeg.chiariglione.org/standards/mpeg-h/high-efficiency-video-coding/requirements-extension-hevc-coding-screen-content>.

[12]

M Parhy, "Game streaming requirement for Future Video Coding", ISO/IEC JTC 1/SC 29/WG 11 Moving Picture Experts Group N36771, Warsaw, Poland, June 2015.

[13]

ITU-T, G Bjontegaard, "Calculation of average PSNR differences between RD-curves", SG 16 VCEG-M33, April 2001,
<https://www.itu.int/wftp3/av-arch/video-site/0104_Aus/>.

[14]

T Daede, A Norkin, and I Brailovskiy, "Video Codec Testing and Quality Measurement", Internet-Draft draft-ietf-netvc-testing-09, January 2020,
<https://tools.ietf.org/html/draft-ietf-netvc-testing-09>.

[15]

Z Wang, E.P. Simoncelli, and A.C. Bovik, "Multiscale structural similarity for image quality assessment", IEEE Thirty-Seventh Asilomar Conference on Signals, Systems and Computers, DOI 10.1109/ACSSC.2003.1292216, November 2003,
<https://ieeexplore.ieee.org/document/1292216>.

[16]

F Bossen, "Common HM test conditions and software reference configurations", Joint Collaborative Team on Video Coding (JCT-VC) of the ITU-T Video Coding Experts Group ( , Document JCTVC-L1100, April 2013,
<http://phenix.it-sudparis.eu/jct/doc_end_user/current_document.php?id=7281>.

[17]

ITU-R, "Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios", ITU-R Recommendation BT.601, March 2011,
<https://www.itu.int/rec/R-REC-BT.601/>.

[18]

ISO/IEC, "Information technology -- Coding of audio-visual objects -- Part 10: Advanced video coding", ISO/IEC DIS 14496-10,
<https://www.iso.org/standard/75400.html>.

[19]

ISO/IEC, "Information technology -- Coding of audio-visual objects -- Part 15: Carriage of network abstraction layer (NAL) unit structured video in the ISO base media file format", ISO/IEC 14496-15,
<https://www.iso.org/standard/74429.html>.

[20]

ITU-R, "Parameter values for the HDTV standards for production and international programme exchange", ITU-R Recommendation BT.709, June 2015,
<https://www.itu.int/rec/R-REC-BT.709>.

Alexey Filippov

Andrey Norkin

Jose Roberto Alvarez