Content for TR 26.998 Word version: 18.1.0

0… 4… 4.2… 4.2.2… 4.2.2.2 4.2.2.3 4.2.2.4 4.2.3… 4.3… 4.4… 4.5… 4.6… 4.6.4… 4.6.5… 4.6.8… 5 6… 6.2… 6.2.4… 6.2.4.2 6.2.5… 6.3… 6.3.4… 6.3.4.2 6.3.5… 6.4… 6.4.4 6.4.5… 6.5… 6.5.4 6.5.5 6.5.6… 6.6… 6.6.4 6.6.5… 7… 8… 8.9 9 A… A.2 A.3… A.4 A.5 A.6 A.7…

6.5.5 Instantiation #2: DCMTSI-based architecture extension with immersive media processing
...

6.5.5 Instantiation #2: DCMTSI-based architecture extension with immersive media processing p. 83

Compared with the instantiation for MTSI-based architecture extension, this instantiation emphasises that the IMS-AGW/MRF may support immersive media processing. It is necessary for 5G EDGAR UEs with poor media capabilities. Figure 6.5.5-1 provides an DCMTSI-based architecture of AR conversational services for EDGAR UE. A 5G EDGAR UE integrated with DCMTSI client in terminal is denoted as an EDGAR-DCMTSI client. An EDGAR-DCMTSI client may request an AR application (i.e., an entry point) via a bootstrap data channel from the data channel server. An EDGAR-DCMTSI client may also generate or retrieve some AR specific data (e.g., pose and viewpoint information) which is transmitted via additional data channels, given that non-media data is handled by using SCTP as specified in IETF RFC 8831. When an EDGAR-DCMTSI client initiates an AR call with another one, the IMS-AGW/MRF with a support of immersive media processing may perform pre-rendering with the media stream originated from the parties of this AR session if they receive the corresponding AR-specific data (i.e. the pose and viewpoint information).

EDGAR-DCMTSI clients negotiate the properties such as reliable or unreliable message transmission, in-order or out-of-order message delivery and an optional protocol for data channel using SDP as defined in IETF RFC 8864. Based on the user plane protocol stack for a basic MTSI client defined in clause 4.2 of TS 26.114 and the Section 6.5 of RFC 8827, all data channels (e.g., both an AR application via bootstrap data channels and AR-specific data via additional data channels) are secured via DTLS.

Copy of original 3GPP image for 3GPP TS 26.998, Fig. 6.5.5-1: DCMTSI-based conversational service architecture for EDGAR UE

Figure 6.5.5-1: DCMTSI-based conversational service architecture for EDGAR UE
(⇒ copy of original 3GPP image)

Furthermore, the IMS-AGW/MRF with a support of immersive media processing are also desirable to 5G STAR UEs due to saving power consumption. Note that the IMS-AGW/MRF with a support of immersive media processing may perform pre-rendering based on the request of the STAR UEs carried in these additional data channels. Particularly, the logical function of immersive media processing may be integrated in the MRF or other media functions.

Figure 6.5.5-2 illustrates the procedure diagram for an immersive AR conversational with two party using EDGAR UEs including an EDGAR-DCMTSI client in the context of the IMS-AGW/MRF with a support of immersive media processing. The procedure is also applicable to establish an immersive AR call where the two parties of a session are STAR UEs or one is STAR UE and the other is EDGAR UE.

Copy of original 3GPP image for 3GPP TS 26.998, Fig. 6.5.5-2: AR-DCMTSI client to AR-DCMTSI client call establishment for EDGAR UE

Figure 6.5.5-2: AR-DCMTSI client to AR-DCMTSI client call establishment for EDGAR UE
(⇒ copy of original 3GPP image)

Assumptions:

AR immersive media may be sent over RTP/UDP/IP and/or SCTP/UDP/IP.
AR immersive media may be negotiated and configured using SDP.
A data channel application may provide rich user experiences by utilizing both user's underlying scene and pose of objects representing users in the scene.

Procedures:

Step 1.

An EDGAR UE initiates a SIP INVITE request, containing the SDP offer with AR media capabilities.

Step 2.

The call propagates to the terminating EDGAR UE.

Step 3.

The terminating EDGAR UE returns an SDP answer in a SIP 183 progress message. The P-CSCF uses the SDP answer to allocate the required resources.

Step 4.

The originating EDGAR UE generate a PRACK which is transited to the terminating side of the call.

Step 5.

The originating EDGAR UE receives an associated 200 OK (PRACK).

Step 6.

The terminating EDGAR UE reserves internal resources to reflect the SDP answer and configures media pipelines.

Step 7.

The originating EDGAR UE sends a SIP UPDATE message with a new SDP offer confirming the selected media parameters.

Step 8.

The 200 OK (UPDATE) response is received for the terminating STAR UE containing the SDP answer.

Step 9.

The terminating EDGAR UE is now alerted and sends a SIP 180 Ringing response.

Step 10.

When the terminating EDGAR UE has answered the call, it sends a 200 OK to the originating EDGAR UE.

Step 11.

The terminating EDGAR UE receives the 200 OK, and sends a SIP ACK message to acknowledge that the call has been established.

Step 12.

The EDGAR UEs retrieve a data channel application for AR through the bootstrap data channel. If the EDGAR UE enables to provide native AR application, this step is not required.

Step 13.

Any additional data channels created and used by the data channel application for AR itself are requested.

Step 14.

The originating EDGAR UE initiates SIP re-INVITE request, containing an updated SDP offer to establish those additional data channels.

Step 15.

The AS/S-CSCF identify an updated SDP offer for additional data channels and modify the "c=" as the IP address of the MRF, and then send this SDP offer to the remote party.

Step 16.

The AS/S-CSCF send this updated SDP offer to the remote party.

Step 17.

The AS/S-CSCF receive an updated SDP answer from the remote party.

Step 18.

The AS/S-CSCF identify this updated SDP answer for additional data channels and modify the "c=" as the IP address of the MRF, and then send this SDP answer to the remote party.

Step 19.

The additional data channels for the data channel application has been established.

Step 20.

The EDGAR UE processes the immersive media to be transmitted.

The AR runtime function captures and processes the immersive media to be sent.
The AR runtime function passes the immersive media data to the AR-DCMTSI client.
The AR-DCMTSI client encodes the immersive media to be transmitted to the IMS-AGW/MRF supporting immersive media processing.

Step 21.

The data channel application for AR collects the AR-specific data, and decides to send them to the AR-DCMTSI client if the AR experiences requires assistance from the network side.

Step 22.

The AR-DCMTSI client sends the AR-specific data (e.g., virtual objects info) to the IMS-AGW/MRF via the designated data channel 1 based on the previous SDP negotiation.

Step 23.

The IMS-AGW/MRF composes, renders and encodes the AR immersive media based on the received media stream and the AR-specific data from the originating party, and finally send them to the terminating party.

Step 24.

The data channel application for AR collects the AR-specific data, and decides to send them to the AR-DCMTSI client if the AR experiences requires assistance from the network side.

Step 25.

The AR-DCMTSI client sends the AR-specific data (e.g. pose info and/or viewport info) to the IMS-AGW/MRF via the designated data channel 2 based on the previous SDP negotiation.

Step 26.

The IMS-AGW/MRF decodes and pre-renders media stream based on the received media stream from the terminating party and the AR-specific data from the originating party, and finally sends them to the originating party.

Step 27.

The EDGAR UE processes the received immersive media.

The AR-DCMTSI client decodes and process the received immersive media.
The AR-DCMTSI client passes the immersive media data to the Lightweight Scene Manager.

The Lightweight Scene Manager renders the immersive media, which includes the registration of the AR content into the real world accordingly.