Content for  TS 26.118  Word version:  18.0.0

This operation targets enhanced 360 video decoding and rendering of H.265/HEVC video for VR applications. Among others, this operation point supports rendering of:

• 4K mono video at up to 120 Hz frame rates
• 3K stereoscopic video at up to 60 Hz frame rates
• Higher than 4K resolutions for restricted coverage
• Rendering of certain viewports in higher quality than others beyond 4K
• ERP and CMP projection
• SDR and HDR transfer characteristics

A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation point shall conform to the requirements in the remainder of clause 5.1.6. A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation point shall support decoding and rendering a Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation point. Detailed receiver requirements are provided in the remainder of clause 5.1.6.

A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation point shall conform to H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6]. Hence, for a Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation point shall comply with the following restrictions:

• The general_profile_idc shall be set to 2 indicating the Main10 profile.
• The general_tier_flag shall be set to 0 indicating the Main tier.
• The value of level_idc shall not be greater than 153 (corresponding to the Level 5.1) and should indicate the lowest level to which the Bitstream conforms.

Bitstreams conforming to the 3GPP VR Flexible H.265/HEVC Operation point shall be encoded with either 8 or 10 bit precision:

• bit_depth_luma_minus8 = 0 or 2 (8 or 10 bits respectively)
• bit_depth_chroma_minus8 = bit_depth_luma_minus8

Receivers conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall support 8 bit and10 bit precision.

Due to the options provided in this operation point, additional original format may be considered that can then be decoded and rendered by a Receiver conforming to this operation point. Recommended original formats beyond those specified in clause 5.1.5.4 for equirectangular projection (ERP) are:

• Mono formats: 8192 × 4096

This operation point permits the distribution of ERP signals directly as well as the conversion of ERP signals to cube-map (CMP) projection. A conversion operation is provided in Annex A.2.3. Typical original cubemap format, either generated by conversion or provided by the content provider, that are suitable for this operation point are listed as follows:

• Mono Formats: 6144x4096, 4608x3072, 4320x2880, 3072x2048, 2880x1920, 2304x1536, 2160x1440
• Stereo Formats with resolution for each eye: 4320x2880, 3072x2048, 2880x1920, 2304x1536, 2160x1440

If original signals are beyond the maximum permitted resolution of the video codec, then region wise packing needs to be applied to generate suitable distribution formats. The distribution formats are more flexible as additional VR metadata as defined in the remainder of clause 5.1.6 may be used. However, for the distribution formats, all requirements of H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6] shall apply to the decoded texture signal. According to H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6], the maximum luminance width and height does not exceed 8,444 pixels. However, for improved interoperability, for a Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation point, the decoded texture signal:

• shall not exceed the luminance width of 8192 pixels, and
• shall not exceed the luminance height of 8192 pixels.

A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall be capable of decoding and rendering Bitstreams with a decoded texture signal of maximum luminance width of 8192 pixels and maximum luminance height of 8192 pixels.

A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall use either Recommendation ITU-R BT.709 [3] colorimetry or Recommendation ITU-R BT.2020 [4] colorimetry in non-constant luminance for standard dynamic range (SDR). For Perceptual Quantization (PQ) High Dynamic Range (HDR), BT.2020 [4] colorimetry in non-constant luminance and the PQ electro-optical transfer function (EOTF) as defined in Recommendation ITU-R BT.2100 [11] are used. For Hybrid Log-Gamma (HLG) High Dynamic Range (HDR), BT.2020 [4] colorimetry in non-constant luminance and the HLG opto-electronic transfer function (OETF) as defined in Recommendation ITU-R BT.2100 [11] are used. Specifically, in the VUI, the colour parameter information shall be present, i.e.:

• video_signal_type_present_flag value and colour_description_present_flag value shall be set to 1.
• If BT.709 [3] is used, it shall be signalled by setting colour_primaries to the value 1, transfer_characteristics to the value 1 and matrix_coeffs to the value 1.
• If BT.2020 [4] and SDR is used,
  • it shall be signalled by setting colour_primaries to the value 9, transfer_characteristics to the value 14 and matrix_coeffs to the value 9,
  • the chroma_loc_info_present_flag should be equal to 1, and if set the chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field shall both be equal to 2
• If BT.2020 [4] and ITU-R BT.2100 [11] are used in HDR,
  • it shall be signalled by setting colour_primaries to the value 9 and matrix_coeffs to the value 9,
  • the chroma_loc_info_present_flag should be equal to 1, and if set, the chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field shall both be equal to 2
• If the PQ EOTF is used, transfer_characteristics shall be set to the value 16.
  • If the HLG OETF is used, transfer_characteristics shall be set to the value 14. The Bitstream shall also contain the alternative_transfer_characteristics SEI message. The alternative_transfer_characteristics SEI message shall be inserted at each RAP, and its parameter preferred_transfer_characteristics shall be set to the value 18.

A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall be capable of decoding and rendering according to any of the above configurations. SEI messages for HDR metadata signalling may be used. The requirements and recommendations for Bitstreams and Receivers, as documented in clause 4.5.5.7 of TS 26.116, also apply for the 3GPP VR Flexible H.265/HEVC Operation Point.

A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall have one of the following frame rates: 24; 25; 30; 24/1001; 30/1001; 50; 60; 60/1001, 90, 100, 120 Hz. Selected combinations of frame rates with other source parameters are provided in Annex A.2.2.2. In the VUI, the timing information may be present:

• If the timing information is present, i.e. the value of vui_timing_info_present_flag is set to 1, then the values of vui_num_units_in_tick and vui_time_scale shall be set according to the frame rates allowed in this clause. The timing information present in the video Bitstream should be consistent with the timing information signalled at the system level.
• The frame rate shall not change between two RAPs. fixed_frame_rate_flag value, if present, shall be set to 1.

There are no requirements on output timing conformance for H.265/HEVC decoding (Annex C of [6]). The Hypothetical Reference Decoder (HRD) parameters, if present, should be ignored by the Receiver. A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall be capable of decoding and rendering Bitstreams that use frame rates according to the bitstream requirements documented above.

For H.265/HEVC random access point (RAP) definition refer to clause 4.5.1.2.1 of TS 26.116. RAPs shall be present in the Bitstream at least once every 5 seconds. It is recommended that RAPs occur in the video Bitstream on average at least every 2 seconds. The time interval between successive RAPs is measured as the difference between their respective decoding time values. If viewport adaptation is offered, then RAPs should occur even more frequently to enable transitioning across these viewport-optimized bitstreams.

Receivers conforming to the 3GPP VR Flexible H.265/HEVC Operation Point should ignore the content of all Video Parameter Sets (VPS) NAL units as defined in Recommendation ITU-T H.265 / ISO/IEC 23008-2 [6]. The following restrictions apply to the active Sequence Parameter Set (SPS):

• The Video Usability Information (VUI) shall be present in the active Sequence Parameter Set. The vui_parameters_present_flag shall be set to 1.
• The chroma sub-sampling shall be 4:2:0, chroma_format_idc value shall be set to 1.
• The source video format shall be progressive, i.e.:
  • The general_progressive_source_flag shall be set to 1,
  • The general_interlaced_source_flag shall be set to 0,
  • The general_frame_only_constraint_flag shall be set to 1.

Receivers conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall support Bitstreams with the restrictions on the SPS defined above.

In addition to the previous constraints on the VUI on colour information in clause 5.1.6.5 and clause 5.1.6.6, this clause contains further requirements. The aspect ratio information shall be present, i.e.:

• The aspect_ratio_present_flag value shall be set to 1.
• The aspect_ratio_idc value shall be set to 1 indicating a square pixel format.

There are no requirements on output timing conformance for H.265/HEVC decoding (Annex C of [6]). The Hypothetical Reference Decoder (HRD) parameters, if present, should be ignored by the Receiver.

This operation point permits using either equirectangular projection or cubemap projection following the MPEG metadata specifications, such the video is automatically rendered in the 3GPP reference system. A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall include at every RAP either:

• the equirectangular projection SEI message (payloadType equal to 150) with the erp_guard_band_flag set to 0, or
• the cubemap projection SEI message (payloadType equal to 151).

A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall be able to process the equirectangular projection SEI message (payloadType equal to 150) and the cubemap projection SEI message (payloadType equal to 151).

This operation point permits to distribute content with less than 360 degree coverage in an encoding optimized manner by the use of region-wise packing. It is recommended that the number of pixels that are projected to non-covered regions are minimized in the decoded texture signal. If this is applied and not the full 360 video is encoded, the region-wise packing SEI message (payloadType equal to 155) shall be included in the bitstream to signal the encoded regions of the 360 video. If present, it shall be present in a H.265/HEVC RAP. Application or system-based signalling may support signalling the exact coverage region in the spherical coordinates.

This operation point permits the use of region-wise packing, for example to optimize the spatial resolution of specific viewports. For some example usage and settings, refer to Annex A.2. A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation Point may include the region-wise packing SEI message (payloadType equal to 155). If present, it shall be present in a H.265/HEVC RAP. A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall be able to process the region-wise packing SEI message (payloadType equal to 155).

A Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation Point may include the frame packing arrange SEI message (payloadType equal to 45). If present, then the following settings shall apply:

• The SEI message is present in a H.265/HEVC RAP.
• The value of frame_packing_arrangement_cancel_flag is equal to 0.
• The value of frame_packing_arrangement_type is equal to 4.
• The value of quincunx_sampling_flag is equal to 0.
• The value of spatial_flipping_flag is equal to 0.
• The value of field_views_flag is equal to 0.
• The value of frame0_grid_position_x is equal to 0.
• The value of frame0_grid_position_y is equal to 0.
• The value of frame1_grid_position_x is equal to 0.
• The value of frame1_grid_position_y is equal to 0.

A Receiver conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall process the frame packing arrangement SEI (payloadType equal to 45) with settings restrictions as above. If processing is supported, then the Receiver shall render the viewport indicated by the message.

For a Bitstream conforming to the 3GPP VR Flexible H.265/HEVC Operation Point:

• the sphere rotation SEI message (payloadType equal to 154) shall not be present.
• any frame-packing arrangement SEI message (payloadType equal to 45) that does not conform to an SEI message defined in clause 5.1.6.13 shall not be present.

Receivers conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall support decoding and displaying 3GPP VR Main H.265/HEVC Operation Point Bitstreams and 3GPP VR Flexible H.265/HEVC Operation Point Bitstreams. Receivers conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall support all Receiver requirements in clause 5.1.6. Specifically, receivers conforming to the 3GPP VR Flexible H.265/HEVC Operation Point shall support decoding and rendering Bitstreams that include the following display or VR rendering metadata:

• the region-wise packing SEI message (for details see clauses 5.1.6.11 and 5.1.6.12),
• the equirectangular projection SEI message (for details see clause 5.1.6.10),
• the cubemap projection SEI message (for details see clause 5.1.6.10),
• the frame-packing arrangement SEI message (for details see clause 5.1.6.13),
• the alternative_transfer_characteristics SEI message with preferred_transfer_characteristics set to the value 18 (for details see clause 5.1.6.5),
• any combinations of those.