Content for TS 26.522 Word version: 18.1.0
4.3 RTP Header Extension for XR Pose
4.3.1 General
4.3.2 SDP Signaling
4.3.3 Header Extension Format
4.4 RTP Header Extension for in-band end-to-end delay measurement
4.4.1 General
4.4.2 One-byte RTP Header Extension Format
4.4.3 Two-byte RTP Header Extension Format
4.4.4 Syntax
4.4.5 Semantics
4.4.6 SDP signaling
...
...
4.3 RTP Header Extension for XR Pose p. 17
4.3.1 General p. 17
An RTP sender that uses RTP to deliver rendered video streams to an RTP receiver should include an RTP HE for XR pose to indicate the XR pose used for rendering the media (rendered pose). The RTP HE for XR pose may be used for signaling either a 6DoF XR pose or a 3DoF XR pose. The RTP HE for XR pose may also be used with audio streams.
The RTP HE for XR pose may also be used by an RTP sender to indicate the XR pose to be rendered to an RTP receiver.
The IANA registration information for the RTP HE for XR pose is provided in Annex D.3.
4.3.2 SDP Signaling p. 17
An RTP client that supports the RTP HE for XR pose shall negotiate the use of the extension using SDP. The signaling of the RTP HE for XR pose shall follow the SDP signaling design, the syntax, and semantics of the "extmap" attribute as outlined in RFC 8285.
For IANA registration, the "reference" field in the registry is 3GPP TS 26.522.
The ABNF syntax for this RTP HE extends the "extmap" attribute as follows:
The extension attribute "3DOF" indicates that the sender uses the RTP HE to signal a 3DoF XR pose, i.e., an XR pose that does not include the position fields x, y, z.
An RTP client that supports the RTP HE for XR pose and receives an SDP offer with "a=extmap" attribute with the URN "urn:3gpp:xr-pose" and the extension attribute "3DOF", shall include the extension attribute "3DOF" in the SDP answer, if the SDP offer is accepted.
The extension attribute "6DOF" indicates that the sender uses the RTP HE to signal a 6DoF XR pose, i.e., an XR pose that includes both the position fields x, y, z and the orientation fields rx, ry, rz, rw.
An RTP client that supports the RTP HE for XR pose and receives an SDP offer with "a=extmap" attribute with the URN "urn:3gpp:xr-pose" and the extension attribute "6DOF", shall include the extension attribute "6DOF" in the SDP answer, if the SDP offer is accepted.
The extension attribute "media" is followed by a list of tokens for "mid" (as defined in RFC 5888) for media streams that can reuse the pose included in the RTP HE. Further details on reuse are provided later in the section.
An RTP endpoint that supports the RTP HE for XR pose and receives an SDP offer with "a=extmap" attribute with the URN "urn:3gpp:xr-pose" shall remove the attribute from the answer for any media that will not use the extension, and retain it for any media that will use it.
4.3.3 Header Extension Format p. 18
If the RTP HE for XR pose is for rendered pose, the RTP sender should use the RTP HE for XR pose to associate the selected pose with the rendered frame. The RTP sender delivers the rendered frames using one or more video streams, depending on the view and projection configuration that is selected by the UE.
If negotiated successfully, an RTP sender should add the RTP HE for XR pose to the RTP stream. The frequency of RTP HE for XR pose shall be at least once in a frame. It may be sent more often but not necessarily in every RTP packet.
The 2-byte (RFC 8285) RTP HE format shall be used for signalling the RTP HE. Format of the HE for 6DoF XR pose is shown below.
If the RTP HE for XR pose is used for signaling a 3DoF XR pose, the fields x, y, z shall be omitted. Format of the HE for 3DoF XR pose is shown below.
The fields rx, ry, rz, rw, x, y, z are defined in single-precision floating-point format (binary32 as per ISO/IEC 60559 [20]).
rx (32 bits):
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0x100 |appbits| length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | L | rx …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ry …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| rz …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| rw …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| x …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| y …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| z …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | XR timestamp …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| XR timestamp continued …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| XR timestamp continued | action_id #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| action_id #2 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0x100 |appbits| length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | L | rx …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ry …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| rz …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| rw …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| | XR timestamp …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| XR timestamp continued …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| XR timestamp continued | action_id #1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| action_id #2 | ... |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
x coordinate of the orientation quaternion of the XR pose.
ry (32 bits):
y coordinate of the orientation quaternion of the XR pose.
rz (32 bits):
z coordinate of the orientation quaternion of the XR pose.
rw (32 bits):
w coordinate of the orientation quaternion of the XR pose.
x (32 bits):
x coordinate of the position of the XR pose in meters.
Y (32 bits):
y coordinate of the position of the XR pose in meters.
z (32 bits):
z coordinate of the position of the XR pose in meters.
XR timestamp (64 bits):
Timestamp for the XR pose. If the RTP HE is used for rendered pose, this timestamp indicates the display time predicted by the XR runtime for the rendered image. Otherwise, this timestamp indicates the associated XR runtime display time for the predicted XR pose. XR timestamp uses the XR system clock and is represented in nanoseconds. The timestamp is passed to the XR runtime together with the rendered swapchain images (e.g. as part of the xrEndFrame call in OpenXR). A receiver may use the XR timestamp together with the RTP timestamp to determine the playout time of the media. XR timestamp shall not be used for media synchronization.
action_id (32 bits):
A list of actions corresponding to the pose x, y, z, rx, ry, rz, rw coordinates. An action_id uniquely identifies an action and it may be an action identifier as defined in the action format of clause 6.2.3 in TS 26.119. The number of action identifiers in one RTP HE for XR pose shall be no more than 10. Hence, the size of the RTP HE is 36+2*n, if a 6DoF XR pose is used, or 24+2*n, if a 3DoF XR pose is used, where n is the number of action identifiers in the HE.
If the RTP HE for XR pose is for rendered pose, the RTP sender should contain an action_id field as defined above, with the list of action identifiers identifying the processed actions for the rendering of the frame.
If the RTP HE for XR pose is for pose to be rendered, the RTP sender should contain an action_id field as defined above, with the list of action identifiers identifying the action for which the pose coordinates apply.
When both video and audio are delivered to an RTP receiver, or when either audio or video is delivered using multiple real-time streams (e.g., left eye + right eye), multiple RTP streams may be associated with the same RTP HE data, e.g., the same pose may have been used for generating multiple streams. This may lead to sending the same RTP HE data multiple times in different streams.
A sender may reuse the XR pose RTP HE of one stream for multiple RTP streams. For example, only the video stream carries the pose RTP HE, but the pose is applicable also for the audio bitstream. In this case, the sender shall include the extension attribute media followed by a space separated list of media ID (MID) values in the "a=extmap" attribute. The MID values indicate all media streams for which the pose RTP HE is applicable to. If the extension attribute media is present, then the media description of all bitstreams that reuse the RTP HE shall include the attribute "mid", as defined in RFC 5888.
4.4 RTP Header Extension for in-band end-to-end delay measurement p. 20
4.4.1 General p. 20
An RTP HE that allows an RTP packet to carry timestamp(s) may help obtain measured delays that are representative of the end-to-end instantaneous delays experienced by the media in the user plane.
Figure 4.4.1-1 shows how the RTP HEs are used to measure the delays, where T1, T2, T3 and T4 are the Originate Timestamp, the Receive Timestamp, the Transmit Timestamp, and the Destination Timestamp, respectively. The one-way delay from the Requester to the Responder is calculated as T2 - T1, the one-way delay in the opposite direction is calculated as T4 - T3, the RTT is calculated as (T4 - T1) - (T3 - T2), and the processing delay on the Responder is calculated as T3 - T2.
The RTP HEs defined below follow RFC 8285.
The IANA registration information for the RTP HE for in-band end-to-end delay measurement is provided in Annex D.4.
4.4.2 One-byte RTP Header Extension Format p. 21
The RTP HE element for the RTP packet that carries only one timestamp T1 is shown below. This is the same as the "RTP Header Extension for Absolute Sender Time" in Annex C.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0xBE | 0xDE | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | L=2 | T1 (24 bits) |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
The RTP HE element for the RTP packet that carries three timestamps T1, T2 and T3 is shown below.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0xBE | 0xDE | length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | L=8 | T1 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| T2 | T3 …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
4.4.3 Two-byte RTP Header Extension Format p. 21
The RTP HE element for the RTP packet that carries one timestamp T1 is shown below.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0x100 |appbits| length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | L=3 | T1 …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|
+-+-+-+-+-+-+-+-+
The RTP HE element for the RTP packet that carries three timestamps T1, T2 and T3 is shown below.
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| 0x100 |appbits| length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| ID | L=9 | T1 …
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| T2 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| T3 |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
4.4.4 Syntax p. 22
T1:
consists of 24 bits, taken from the 6 LSB bits of the integer part and the 18 MSB bits of the fractional part of the NTP timestamp format defined in RFC 5905.
T2:
follows the syntax of T1.
T3:
follows the syntax of T1.
4.4.5 Semantics p. 22
T1:
Originate Timestamp. It represents the time when the Requester transmits the RTP packet toward the Responder.
T2:
Receive Timestamp. It represents the time when the Responder receives the RTP packet that carries the Originate Timestamp T1.
T3:
Transmit Timestamp. It represents the time when the Responder transmits the RTP packet that carries the Originate Timestamp T1, the Receive Timestamp T2, and the Transmit Timestamp T3.
4.4.6 SDP signaling p. 22
The signaling of the delay measurement RTP HE shall follow the SDP signaling design and the syntax and semantics of the "extmap" attribute as outlined in RFC 8285.
For the RTP HE carrying only T1, the ABNF syntax for the "extmap" attribute is as follows:
If the extensionattributes is absent, the RTP HE follows the one-byte format, i.e., the "short" format. If extensionattributes is "short", the RTP HE follows the one-byte format. If extensionattributes is "long", the RTP HE follows the two-byte format.
Below is an example (Example 1):
For the RTP HE carrying T1, T2 and T3, the ABNF syntax for the "extmap" attribute is as follows:
The extension attributes have the following semantics:
In the example,
- dependent-extmap-ID: identifies the Requester sent RTP HE (i.e., carrying T1) on which the Responder sent RTP HE (i.e., carrying T1, T2 and T3) depends. Timestamps T1 and T2 included in the Responder sent RTP HE are the time the Requester sent RTP HE is transmitted at and the time the Requester sent RTP HE is received at the Responder, respectively.
- processing-ID: identifies a processing module on the Responder which takes data carried in RTP packets with the RTP HE identified by dependent-extmap-ID, processes them and produces data that are then carried in RTP packets with this RTP HE.
- m-line-label: is the SDP "label" attribute defined in RFC 4574, and it identifies a media stream from the Requester to the Responder and associates the RTP HE in that media stream to this RTP HE.
a=extmap:5 urn:3gpp:delay-measurement-response:rel-18 short dependent-extmap-ID=4;dependent-rtp-he-m-line-label=2;processing-ID=7
- 5 is the RTP HE ID
- 4 is the value of the attribute dependent-extmap-ID, which is the RTP HE ID of the RTP HE in Example 1. This establishes a binding between the two RTP HEs.
- 7 is the processing-ID.
- 2 is the SDP "label" attribute that identifies the media stream corresponding to "a=label:2" in the SDP signaling, and the RTP packets from the media stream are used for the binding.
