Content for  TS 26.118  Word version:  18.0.0

This operation targets enhanced 360 video decoding and rendering of H.265/HEVC video for VR applications. Among others, this operation point supports among others rendering of:

• 4K mono video at up to 60 Hz frame rates
• 3K stereoscopic video at up to 60 Hz frame rates
• Higher than 4K resolutions for restricted coverage
• Rendering of certain viewports in higher quality than others beyond 4K
• extended colour space and SDR transfer characteristics

A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation point shall conform to the requirements in the remainder of clause 5.1.5. A Receiver conforming to the 3GPP VR Main H.265/HEVC Operation point shall support decoding and rendering a Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation point. Detailed receiver requirements are provided in the remainder of clause 5.1.5.

A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation point shall conform to H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6]. Hence, for a Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation point shall comply with the following restrictions:

• The general_profile_idc shall be set to 2 indicating the Main10 profile.
• The general_tier_flag shall be set to 0 indicating the Main tier.
• The value of level_idc shall not be greater than 153 (corresponding to the Level 5.1) and should indicate the lowest level to which the Bitstream conforms.

Bitstreams conforming to the 3GPP VR Main H.265/HEVC Operation point shall be encoded with either 8 or 10 bit precision:

• bit_depth_luma_minus8 = 0 or 2 (8 or 10 bits respectively)
• bit_depth_chroma_minus8 = bit_depth_luma_minus8

Receivers conforming to the 3GPP VR Main H.265/HEVC Operation Point shall support 8 bit and10 bit precision.

Due to the options provided in this operation point, additional original format may be considered that can then be decoded and rendered by a Receiver conforming to this operation point. Recommended original formats beyond those specified in clause 5.1.4.3 for equirectangular projection (ERP) are:

• Mono formats: 6144 × 3072, 5880 × 2880
• Stereo formats with resolution for each eye: 3840 × 1920, 2880 × 1440, 2048 × 1024

If original signals are beyond the maximum permitted resolution of the video codec, then the region-wise packing needs to be applied to generate suitable distribution formats. The distribution formats are more flexible as additional VR metadata as defined in the remainder of clause 5.1.5 may be used. However, for the distribution formats, all requirements of H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6] shall apply to the decoded texture signal. According to H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6], the maximum luminance width and height does not exceed 8,444 pixels. In addition to the H.265/HEVC Main-10 Profile Main Tier Profile Level 5.1 [6] constraints, a Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation point, the decoded texture signal shall in addition:

• not exceed the luminance width of 8192 pixels, and
• not exceed the luminance height of 8192 pixels.

A Receiver conforming to the 3GPP VR Main H.265/HEVC Operation Point shall be capable of decoding and rendering Bitstreams with a decoded texture signal of maximum luminance width of 8192 pixels a, maximum luminance height of 8192 pixels and the overall profile/level constraints.

A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation Point shall use either Recommendation ITU-R BT.709 [3] colorimetry or Recommendation ITU-R BT.2020 [4] colorimetry in non-constant luminance for standard dynamic range (SDR). Specifically, in the VUI, the colour parameter information shall be present, i.e.:

• video_signal_type_present_flag value and colour_description_present_flag value shall be set to 1.
• If BT.709 [3] is used, it shall be signalled by setting colour_primaries to the value 1, transfer_characteristics to the value 1 and matrix_coeffs to the value 1.
• If BT.2020 [4] and SDR is used,
  • it shall be signalled by setting colour_primaries to the value 9, transfer_characteristics to the value 14 and matrix_coeffs to the value 9;
  • the chroma_loc_info_present_flag should be equal to 1, and if set, the chroma_sample_loc_type_top_field and chroma_sample_loc_type_bottom_field shall both be equal to 2.

A Receiver conforming to the 3GPP VR Main H.265/HEVC Operation Point shall be capable of decoding and rendering according to any of the two above configurations.

A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation Point shall have one of the following frame rates: 24; 25; 30; 24/1001; 30/1001; 50; 60; 60/1001 Hz. Selected combinations of frame rates with other source parameters are provided in Annex A.2.2.2. In the VUI, the timing information may be present:

• If the timing information is present, i.e. the value of vui_timing_info_present_flag is set to 1, then the values of vui_num_units_in_tick and vui_time_scale shall be set according to the frame rates allowed in this clause. The timing information present in the video Bitstream should be consistent with the timing information signalled at the system level.
• The frame rate shall not change between two RAPs. fixed_frame_rate_flag value, if present, shall be set to 1.

There are no requirements on output timing conformance for H.265/HEVC decoding (Annex C of [6]). The Hypothetical Reference Decoder (HRD) parameters, if present, should be ignored by the Receiver. A Receiver conforming to the 3GPP VR Main H.265/HEVC Operation Point shall be capable of decoding and rendering Bitstreams that use frame rates according to the bitstream requirements documented above.

For H.265/HEVC random access point (RAP) definition refer to clause 4.5.1.2.1 of TS 26.116. RAPs shall be present in the Bitstream at least once every 5 seconds. It is recommended that RAPs occur in the video Bitstream on average at least every 2 seconds. The time interval between successive RAPs is measured as the difference between their respective decoding time values. If viewport adaptation is offered, then RAPs should occur even more frequently to enable transitioning across these viewport-optimized bitstreams.

Receivers conforming to the 3GPP VR Main H.265/HEVC Operation Point should ignore the content of all Video Parameter Sets (VPS) NAL units as defined in Recommendation ITU-T H.265 / ISO/IEC 23008-2 [6]. The following restrictions apply to the active Sequence Parameter Set (SPS):

• The Video Usability Information (VUI) shall be present in the active Sequence Parameter Set. The vui_parameters_present_flag shall be set to 1.
• The chroma sub-sampling shall be 4:2:0, chroma_format_idc value shall be set to 1.
• The source video format shall be progressive, i.e.:
  • The general_progressive_source_flag shall be set to 1,
  • The general_interlaced_source_flag shall be set to 0,
  • The general_frame_only_constraint_flag shall be set to 1.

Receivers conforming to the 3GPP VR Main H.265/HEVC Operation Point shall support Bitstreams with the restrictions on the SPS defined above.

In addition to the previous constraints on the VUI on colour information in clause 5.1.5.5 and clause 5.1.5.6, this clause contains further requirements. The aspect ratio information shall be present, i.e.:

• The aspect_ratio_present_flag value shall be set to 1.
• The aspect_ratio_idc value shall be set to 1 indicating a square pixel format.

There are no requirements on output timing conformance for H.265/HEVC decoding (Annex C of [6]). The Hypothetical Reference Decoder (HRD) parameters, if present, should be ignored by the Receiver.

This operation point permits using either equirectangular projection following the MPEG metadata specifications, such the video is automatically rendered in the 3GPP reference system. A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation Point shall include at every RAP the equirectangular projection SEI message (payloadType equal to 150) with the erp_guard_band_flag set to 0.

This operation point permits to distribute content with less than 360 degree coverage in an encoding optimized manner by the use of region-wise packing. It is recommended that the number of pixels that are projected to non-covered regions are minimized in the decoded texture signal. If this is applied and not the full 360 video is encoded, the region-wise packing SEI message (payloadType equal to 155) shall be included in the bitstream to signal the encoded regions of the 360 video. If present, it shall be present in a H.265/HEVC RAP. Application or system-based signalling may support signalling the exact coverage region in the spherical coordinates.

This operation point permits the use of region-wise packing, for example to optimize the spatial resolution of specific viewports. For some example usage and settings, refer to Annex A.2. A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation Point may include the region-wise packing SEI message (payloadType equal to 155). If present, it shall be present in a H.265/HEVC RAP. A Receiver conforming to the 3GPP VR Main H.265/HEVC Operation Point shall be able to process the region-wise packing SEI message (payloadType equal to 155).

A Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation Point may include the frame packing arrange SEI message (payloadType equal to 45). If present, then the following settings shall apply:

• The SEI message is present in a H.265/HEVC RAP.
• The value of frame_packing_arrangement_cancel_flag is equal to 0.
• The value of frame_packing_arrangement_type is equal to 4.
• The value of quincunx_sampling_flag is equal to 0.
• The value of spatial_flipping_flag is equal to 0.
• The value of field_views_flag is equal to 0.
• The value of frame0_grid_position_x is equal to 0.
• The value of frame0_grid_position_y is equal to 0.
• The value of frame1_grid_position_x is equal to 0.
• The value of frame1_grid_position_y is equal to 0.

A Receiver conforming to the 3GPP VR Main H.265/HEVC Operation Point shall process the frame packing arrangement SEI (payloadType equal to 45) with settings restrictions as above. If processing is supported, then the Receiver shall render the viewport indicated by the message.

For a Bitstream conforming to the 3GPP VR Main H.265/HEVC Operation Point:

• the sphere rotation SEI message (payloadType equal to 154) shall not be present.
• any frame-packing arrangement SEI message (payloadType equal to 45) that does not conform to an SEI message defined in clause 5.1.5.13 shall not be present.

Receivers conforming to the 3GPP VR Main H.265/HEVC Operation Point shall support decoding and displaying 3GPP VR Main H.265/HEVC Operation Point Bitstreams. Receivers conforming to the 3GPP VR Main H.265/HEVC Operation Point shall support all Receiver requirements in clause 5.1.5. Specifically, receivers conforming to the 3GPP VR Main H.265/HEVC Operation Point shall support decoding and rendering Bitstreams that include the following VR rendering metadata:

• the region-wise packing SEI message (for details see clauses 5.1.5.11 and 5.1.5.12)
• the equirectangular projection SEI message (for details see clause 5.1.5.10)
• the frame-packing arrangement SEI message (for details see clause 5.1.5.13)
• any combinations of those