Tech-invite  3GPPspace  IETFspace
21  22  23  24  25  26  27  28  29  31  32  33  34  35  36  37  38  4‑5x

Content for  TR 26.928  Word version:  18.0.0

Top   Top   Up   Prev   Next
0…   4…   4.1.2…   4.2…   4.3…   4.4…   4.5…   4.6…   4.6.7   4.7…   4.9…   5…   6…   7…   8   A…   A.4…   A.7…   A.10…   A.13   A.14   A.15   A.16   A.17   A.18… 
A.4 Use Case 3: Streaming of Immersive 6DoF  A.5 Use Case 4: Emotional Streaming  A.6 Use Case 5: Untethered Immersive Online Gaming 
...

 

A.4  Use Case 3: Streaming of Immersive 6DoF  p. 86

Use Case Description:
Streaming of Immersive 6DoF
Alice consumes a recorded highlight of a basketball match being seated close to the court by using an application on the 5G enabled HMD. For this, Alice wears an HMD together with a 6DoF manual controller. The HMD is connected to a 5G network but has no other tethered connection. The 6DoF controller allows to change the viewing position (i.e. the seat) and looking at the action from different angles. In addition, restricted local 6DoF movement of Alice at a location enables to locally interact with the scene based on HMD sensors. Even more the controller allows to rewind, slow mo and pause the scene. In the pause or slow-motion mode, the scene can be viewed from different angles using the controller and head motion. The scene is overlaid with information that helps Alice to navigate through the scene. Alice feels present in the scene.
In an extension to the use case, the game is consumed in a live mode.
Categorization
Type:
VR
Degrees of Freedom:
3DoF+, 6DoF
Delivery:
Streaming, Interactive, Split
Device:
HMD with a controller
Preconditions
  • Application is installed that permits to consume the scene
  • The application uses existing HW capabilities on the device, including A/V decoders, rendering functionalities as well as sensors. Inside-out Tracking is available.
  • Media is captured properly and accessible on a server, preferably on a CDN.
Requirements and QoS/QoE Considerations
  • Required QoS:
    • Bitrates and Latencies that are sufficient to render the viewport within the immersive limits.
    • Some numbers are provided here: https://www.roadtovr.com/nextvr-latest-tech-is-bringing-new-levels-of-fidelity-to-vr-video/
      • in the best case scenario with 8 Mbps bandwidth, the company can now stream 20 pixels per degree. Keep in mind, that's also in stereo and at 60 FPS
      • plans to roll out this higher-res playback
    • If full 6DoF with presence needs to be enabled, up to 100 Mbit/s may be necessary.
    • However, with viewport adaptive streaming and/or split rendering architectures, the requirements on bitrates may be lower, but the latency requirements may increase. A more detailed study is necessary.
  • Required QoE:
    • Fast startup of the service,
    • fast reaction to manual controller information,
    • reaction to head and limited body movement within immersive limits,
    • seamless experiences when moving across positions
    • providing sufficient AV experience to enable presence. https://xinreality.com/wiki/Presence
      • highest image quality, stereoscopy
      • should also work in slow motion
Feasibility
Content generated in 6DoF
Selected Devices/XR Platforms supporting this:
Potential challenges to make this happen within 3 years
  • Broadly available high-quality 6DoF and volumetric capturing systems. There are still not enough variety of volumetric content to get a feel for how it would handle more challenging scenes like those with closer and/or faster moving objects
  • Broad availability of HMDs and end devices supporting the playback
  • Availability of access bandwidth to stream such services
Potential Standardization Status and Needs
The following aspects may require standardization work:
  • Coded Representation of Audio/Video Formats as well as geometry data
  • Scene composition and description
  • Storage and Cloud Access Formats
  • Delivery Architectures to support 6DoF Streaming
  • Content Delivery and Streaming Protocols
  • Decoding, rendering and sensor APIs
  • Network conditions that fulfill the QoS and QoE Requirements
Up

A.5  Use Case 4: Emotional Streaming  p. 88

Use Case Description:
Emotional Streaming
Bob is watching a horror movie using a 5G connected HMD. He is fascinated, but his body reaction, eye rolling, and other attributes are collected and are used to create a personalized story line. Movie effects are adjusted for personal preferences while reactions are collected when watching the movie. Bob's emotional reactions determine the story-line.
Alice is watching the same story on her newest 5G connected smart phone.
Categorization
Type:
2D interactive, VR and AR
Degrees of Freedom:
2D, 3DoF+, 6DoF
Delivery:
Streaming, Interactive, Split
Device:
Phone and HMD
Preconditions
  • Application is installed that permits to consume the story
  • The application uses existing HW capabilities on the device, including A/V decoders, rendering functionalities as well as sensors
  • The application uses AI functionalities to extract personalized reactions based on sensor tracking
Requirements and QoS/QoE Considerations
  • QoS:
    • Bitrates and Latencies that are sufficient to render the viewport within the immersive limits or at least to react to the emotions
  • QoE:
    • Sufficiently fast reaction to body emotion feedback,
    • for HMD, reaction to head movement within immersive limits,
    • providing sufficient AV experience to enable presence.
    • Streaming with seamless transitions from one scene to either of the choices
Feasibility
https://www.cnet.com/news/with-5g-you-wont-just-be-watching-video-itll-be-watching-you-too/
Interactive and branching content
Device Features
  • Facial expression tracking with AI is available on mobile devices
  • Eye Tracking combined with AI is available on mobile devices
  • IoT/Wearable devices provide the ability to measure biometric metrics such as heart beat and other stress detecting factors (skin changes, etc.) and may be connected with app
Biometric and Emotion Tracking Technologies are summarized:
Potential Standardization Status and Needs
The following aspects may require standardization work:
  • Coded Representation of Audio/Video Formats
  • Seamless splicing and smooth transitions across storylines
  • Scene composition and description
  • Storage and Cloud Access Formats
  • Content Delivery Protocols
  • Decoding, rendering, sensor and emotion tracking APIs
  • Biometrics and Emotion Metadata definition and delivery
Up

A.6  Use Case 5: Untethered Immersive Online Gaming  p. 89

Use Case Description:
Untethered Immersive Online Gaming
100 friends play Fortnite Battle Royal ™. Of the the 100 friends, several are on travel and connect on a stand-alone HMD. The HMD has a with 5G connection. Fortnite Battle Royale ™ is a free-to-play battle royale video game. As a battle royale game, Fortnite Battle Royale features up to 100 players, alone, in duos, or in squads of up to four players, attempting to be the last player or group alive by killing other players or evading them, while staying within a constantly shrinking safe zone to prevent taking lethal damage from being outside it. Players start with no intrinsic advantages, and scavenge for weapons and armor to gain the upper hand on their opponents. The game features cross-platform play between the platforms that was limited for the first five seasons, before the restrictions were eased.
Other popular VR games are here:
Categorization
Type:
VR
Degrees of Freedom:
6DoF
Delivery:
Streaming, Interactive, Split
Device:
HMD with a Gaming controller
Preconditions
  • Gaming client is installed that permits to consume the game
  • The application uses existing HW capabilities on the device, including game engines, rendering functionalities as well as sensors. Inside-out Tracking is available.
  • Connectivity to the network is provided.
Requirements and QoS/QoE Considerations
  • Collected Statistics:
  • Required QoS:
    • https://broadbandnow.com/guides/best-internet-service-setup-serious-gamers
      • Any connection over 2 mbps with less than 75ms ping should work well for 99% of games.
      • the main factors affecting your gameplay are:
        • Efficiency of your network
        • Distance to other players in multiplayer games
      • QoS and network prioritization might not matter much for the average Internet user, but for gamers it can make a big difference in network lag.
      • Ping is king.
    • Different scenarios need to be looked at, for example where the rendering is happening.
  • Required QoE:
    • fast reaction to manual controller information,
    • reaction to head movement within immersive limits,
    • providing sufficient gaming rendering experience to enable presence.
    • supporting frame rate not lower than 60 FPS and resolution not lower than 8K
The TR 22.842 provides some information as well, please refer to clause 5.3.1. Summary of some discussions:
  • Latency requirements for online games may be very tight. Examples
    • Current mainstream FPS (First Person Shooter) game requires 60 frames per second, which means frame interval is 16.67ms. If rendering is done in the cloud and taking out the delay for rendering and encoding/decoding processing, the network round trip time (RTT) delay should be less than 5ms.
    • MOBA (Multiplayer Online Battle Arena) game requires 20ms RTT.
  • Resolutions and frame rates need to be sufficiently high: higher than 60 FPS and 8K resolution
  • Packet loss rates should be low as game experiences degrade quickly
And some references from TR 22.842
  • O. Abari, D. Bharadia, A. Duffield, and D. Katabi, "Cutting the Cord in Virtual Reality," in Proceedings of the 15th ACM Workshop on Hot Topics in Networks. ACM, 2016, pp. 162-168.
  • E. Bastug, M. Bennis, M. M´dard, and M. Debbah, "Toward Interconnected Virtual Reality: Opportunities, Challenges, and Enablers," IEEE Communications Magazine, vol. 55, no. 6, pp. 110-117, 2017.
  • Athul Prasad, Mikko A. Uusitalo, David Navrátil, and Mikko Säily, "Challenges for Enabling Virtual Reality Broadcast Using 5G Small Cell Network", IEEE Wireless Communications and Networking Conference Workshops, pp. 220-225, 2018.
  • Mohammed S. Elbamby, Cristina Perfecto, Mehdi Bennis, and Klaus Doppler, "Toward Low-Latency and Ultra-Reliable Virtual Reality", IEEE Network, March/April, 2018.
  • Orlosky, Jason & Kiyokawa, Kiyoshi & Takemura, Haruo, "Virtual and Augmented Reality on the 5G Highway", Journal of Information Processing, 25. 133-141. 10.2197/ipsjjip.25.133.
  • J. Huang, Z. Chen, D. Ceylan and H. Jin, "6-DOF VR videos with a single 360-camera", 2017 IEEE Virtual Reality (VR), Los Angeles, CA, 2017, pp. 37-44. doi: 10.1109/VR.2017.7892229.
  • Impact of Packet Losses on the Quality of Video Streaming, https://www.diva-portal.org/smash/get/diva2:831420/FULLTEXT01.pdf
  • New Study from GSMA and CAICT Forecasts That China Will Be the World's Largest 5G Market by 2025, https://www.webscalenetworking.com/news/2017/06/27/8571173.htm
Feasibility
Available Games
Selected Devices/XR Platforms supporting this:
An important aspect is that the processing power of untethered devices is typically lower as all processing needs to be done on the device. The feasibility is likely improved by supporting the device with additional network processing.
Demos and Architectures are provided that show cloud and split rendering:
Potential Challenges:
  • Getting end-to-end workflow in place
  • Operational costs
Potential Standardization Status and Needs
The following aspects may require standardization work:
  • Network conditions that fulfill the QoS and QoE Requirements
  • Content Delivery Protocols
  • Decoding, rendering and sensor APIs
  • Architectures for computing support in the network
  • TR 22.842 provides a gap analysis in clause 5.3.6 that is in line with these needs
Up

Up   Top   ToC