Content for  TR 22.837  Word version:  19.4.0

Extended Reality (XR) is an important 5G use case. Split-rendering architectures, where the heavy XR video rendering computation is done at the application server based on control information received from the UE, poses strict Quality-of-Service (QoS) requirements in terms of round-trip latency and throughput for delivering the video and control info. It is therefore crucial to always maintain a high-quality wireless link for XR. Thus, it is critical to predict and adapt fast to wireless channel changes. This is especially true in Millimeter wave bands in which the channel and propagation characteristics are very sensitive to user and environment changes such as blockages, user motion or rotation. To adapt fast to the wireless channel changes, an understanding of the wireless channel dynamics is required. The channel dynamics depend on understanding the surrounding environment such as the transmitter and receiver locations, geometry of the buildings, moving scatterers, location and material of blockers, etc. Interestingly, most of the XR streaming devices (e.g., 5G phones, AR/VR headsets) and third-party entities that support 5G (i.e., 3GPP sensors) also support non-3GPP sensors, such as RF sensors, Inertial Measurement Units (IMU) sensors, RGB cameras, position sensors, and others. In light of the availability of 3GPP and non-3GPP sensors and the need of environment understanding, it is therefore natural to utilize the overall sensing information to acquire an understanding of the surrounding environment. To this end, a "Sensing RF Map Service" can be envisioned that enables the collection of sensing information from 3GPP and non-3GPP sensors, process and provide that information to a sensing service.

• The input to this "Sensing RF Map" service could be 3GPP sensing data and non-3GPP sensing data from multiple sensors, e.g., RF sensing data, XR user position, camera images, depth maps, hand tracking, motion type, etc. Such input could be produced by a 5GS entity (e.g,, UE or RAN entities) or by a third-party (e.g., surveillance camera). It is essential to note that the collection of this sensing should be done with appropriate user consent and adherence to regional and national regulations.
• The processing of 3GPP and non-3GPP sensing data can be performed within the 5G system or outside the 5GS (for example on an application server). In this use case, we are focus on processing in the 5G system.
• The output of this service (i.e., sensing result) is some understanding of the environment and/or impact to communication performance of a service consumer, e.g., RF environment mapping, etc. When sensing result is shared outside of the 5GS, the appropriate consent and permissions for sharing this information is required.
• The consumer of this service could be a third-party application or other entities in the 5GS.

It is important to note that while this service is provided by 5GSor edge server, business model for the monetization of this service would need to consider factors such as the entities involved in sensing, the transfer of the non-3GPP sensing data and the value of sensing RF Map information produced by these entities as well as the value to the consumer of the service. These considerations are also required for scenarios involving 3GPP only sensing operations but additional considerations are indeed required for non-3GPP sensing data which is generated outside the 5G system.

Jose is playing a game inside a gaming arena using a VR headset that is connected to a RAN entity. The VR headset, RAN entity and third-party surveillance system are configured to provide "3GPP sensing data and non-3GPP sensing data" to the "Sensing RF Map Service". The VR headset is equipped with 3GPP sensors and non-3GPP sensors such as, IMU sensors and cameras, and it can provide sensing inputs e.g., 3GPP sensing data, headset pose and location, velocity, images of the environment and processed images (such as motion pattern and maps). Also, the VR headset can provide communication reference signal measurements or reports to the Sensing RF Map Service. RAN entity has 3GPP NR RF capabilities and can provide 3GPP sensing data to the 5GS, which processes and provides sensing results to the sensing RF Map service. The gaming arena also has cameras deployed by a trusted third-party surveillance camera company and can provide images of the environment and processed images (such as motion pattern and maps) to the 5GS.

1. Jose is playing a game using a VR headset in an arena with some obstacles and other gamers in the environment. Jose moves through the arena and approaches a communication blocker which could potentially impact the performance of the wireless communication between VR headset and the RAN entity.
2. Jose's VR headset, RAN entity and the third-party surveillance system provide 3GPP sensing data, and non-3GPP sensing data to the Sensing RF Map Service. The Sensing RF Map Service combines the 3GPP and non-3GPP sensing data to produce a sensing result which is a comprehensive RF map of the environment surrounding the headset (e.g. information such as the location of RAN entities, reflectors, static blockers, etc. and an indication of wireless link blockage event, e.g., people walking by blocking the 5G link).
3. 5GS uses the RF map to predict that Jose's communication link is about to be blocked if he comes close to the blocker and such prediction is sent to communication and/or the application layers of the game. For example, the application layer adjusts the content of rendered video frames accordingly (e.g., lowers the frame rate, adds a virtual obstacle in the rendered video to prevent Jose from coming close to the blocker.)

Jose enjoys seamless XR gaming application without video frame drops, i.e., no video glitches. This is because Sensing RF map Information was leveraged to assist both the communication service as well as the application.

None.

[PR 5.21.6-1] [PR 5.21.6-2] [PR 5.21.6-3] [PR 5.21.6-4]