Content for TR 26.865 Word version: 18.0.0
4.1.3.4 Scenario 4 - Decoding and Pre-Rendering on Cloud/Edge, further decoding, and Pre-Rendering on 5G UE
4.1.3.4.1 Variation A - Pose Estimation, Post-Rendering and audio playback on End Device
4.1.3.4.2 Variation B - Pose Estimation, Post-Rendering and audio playback on End Device
4.1.3.4.3 Variation B.1 - Pose Estimation on End Device, audio decoding and head-tracked rendering (ISAR) and audio playback on TWS Earbuds/Headphones
4.1.3.4.4 Variation C - Pose Estimation, Post-Rendering and audio playback on TWS Earbuds/Headphones
...
...
4.1.3.4 Scenario 4 - Decoding and Pre-Rendering on Cloud/Edge, further decoding, and Pre-Rendering on 5G UE p. 13
This scenario is as follows in Figure 4.1-15, referring to a type of "5G EDGe-Dependent AR (EDGAR) UE" in TR 26.998, with the main characteristic that the binaural rendering process is shared between the Cloud/Edge, the 5G UE and the End Device (e.g. type 1 or 2). The 5G UE connects to Cloud/Edge through an embedded 5G modem, the 5G UE and End Device connect through WiFi or 5G sidelink, maybe through Bluetooth for audio. End Device sends Pose Information to Cloud/Edge if needed, and the Cloud/Edge, 5G UE and End Device provide the capabilities of decoding and rendering together. More specifically, immersive audio decoding, first pre-rendering, and re-encoding is performed in Cloud/Edge. The first pre-rendering may be done using 6 DOF pose information. The re-encoding may be using a first intermediate ISAR format, e.g., still supporting 3 DOF head-tracked rendering. The 5G UE features the combination of a first ISAR decoder and second ISAR pre-renderer. The re-encoding may be using a second intermediate ISAR format supporting 3 DOF head-tracked binaural post-rendering. The 5G UE also relays pose information to the Core/Edge. The End Device features a decoder, post-renderer and a pose estimator. Motion to sound latency can at least be partially compensated, since the End Device and 5G UE can jointly provide pose correction and head-tracked binaural rendering.
Figure 4.1-15: 5G Cloud/Edge dependent End Device with decoding and post-rendering capabilities, 5G UE and Cloud/Edge performing ISAR Pre-Rendering jointly (see note in clause 0)
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
4.1.3.4.1 Variation A - Pose Estimation, Post-Rendering and audio playback on End Device p. 13
In Variation A, as depicted in Figure 4.1-16, the End Device features an ISAR decoder and post-renderer, built-in loudspeakers for binaural audio playback, and a pose estimator.
Figure 4.1-16: 5G Cloud/Edge dependent End Device with decoding and post-rendering capabilities and built-in loudspeakers, 5G UE and Cloud/Edge performing ISAR Pre-Rendering jointly
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
4.1.3.4.2 Variation B - Pose Estimation, Post-Rendering and audio playback on End Device p. 14
In Variation B, as depicted in Figure 4.1-17, a pair of TWS Earbuds/Headphones is used to playback the binaural audio instead of the built-in speakers used in Variation A. The End Device performs pose estimation, ISAR decoding and head-tracked binaural post-rendering followed by stereo re-encoding the binaural audio signal. The pose information is sent to the 5G UE where it is used in the ISAR decoding and rendering and ISAR re-encoding stage. In addition, it is relayed to the Cloud/Edge. The TWS Earbuds/Headphones decode the binaural audio signal and perform audio playback. Variation B is expected to be more prevalent than Variation C described below due to possibly better pose estimation capability by the End Device.
Figure 4.1-17: 5G UE-dependent End Device with Pose Estimator, ISAR decoder & post-renderer and stereo re-encoder, with connected TWS Earbuds/Headphones, 5G UE and Cloud/Edge performing ISAR Pre-Rendering jointly
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
4.1.3.4.3 Variation B.1 - Pose Estimation on End Device, audio decoding and head-tracked rendering (ISAR) and audio playback on TWS Earbuds/Headphones p. 14
Variation B.1, as depicted in Figure 4.1-18, is like Variation B, except that the TWS Earbuds/Headphones are ISAR Decoder capable. The End Device relays the coded audio (ISAR format) from the 5G UE to the TWS Earbuds/Headphones and provide pose information to the TWS Earbuds/Headphones. Alternatively, the 5G UE can pass the coded audio directly to the TWS Earbuds/Headphones.
Figure 4.1-18: 5G UE-dependent End Device with Pose Estimator, ISAR decoder & post-renderer and stereo re-encoder, with connected TWS Earbuds/Headphones, 5G UE and Cloud/Edge performing ISAR Pre-Rendering jointly
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
4.1.3.4.4 Variation C - Pose Estimation, Post-Rendering and audio playback on TWS Earbuds/Headphones p. 14
In Variation C, as depicted in Figure 4.1-19, TWS Earbuds/Headphones perform ISAR decoding and head-tracked binaural post-rendering of audio and playback binaural audio. In addition, they perform pose estimation and provide pose information to the End Device or directly to the 5G UE. The End Device may be used to relay pose information and coded audio between TWS Earbuds/Headphones and the 5G UE. The 5G UE uses the pose information in the ISAR decoding and rendering and ISAR re-encoding stage. In addition, it is relayed to the Cloud/Edge.
Figure 4.1-19: 5G UE-dependent End Device with connected TWS Earbuds/Headphones featuring Pose Estimator, 5G UE and Cloud/Edge performing ISAR Pre-Rendering jointly
(⇒ copy of original 3GPP image)
(⇒ copy of original 3GPP image)
