Content for  TR 22.856  Word version:  19.2.0

Operating vehicles, lifting devices, or machines in an industrial environment is hazardous when achieved manually and locally by a human. Depending on the environment, operators are exposed to dangerous material, toxic fumes, extreme temperatures, landslide risks, radioactivity, etc. AGVs already exist, although it is expected that human operators can take remote control to remotely operate such moving vehicles. In this use case, it is proposed to leverage 5G to provide an end-to-end system in which a remote user controls a moving device (vehicle, lifting device, robot, etc.) with an immersive cockpit displayed on a virtual reality head-mounted display and haptic gloves for control. Furthermore, the cockpit is complemented with information from the digital twin of the place in where the user operates (e.g., sensors in a factory, type of material around, other moving vehicles or persons). The use case improves user safety and makes the operations even more accurate by merging additional information from a digital twin.

Bob works in a seaport; he operates a lifting device. The place in which he is operating is surrounded by cranes, machines, containers, pipes, and barrels containing hazardous substances. A new mobile metaverse service is available: instead of locally controlling the device, Bob is installed in a safe remote location from which he is working. The surrounding information is available through a digital twin of the seaport and can come from various sources (IoT sensors, CCTV cameras, connected machines, and other vehicles). In order to maximize Bob's efficiency, the metaverse service experience delivered by the system is real-time with non-noticeable latency. This use case includes both location related and location agnostic service experience examples. The mobile metaverse service Bob uses for teleoperation is running on a mobile metaverse server. In addition, Bob is equipped with a head-mounted display and haptic gloves to remotely control the vehicle.

1. This morning, Bob stayed home as his boss informed him about a potential hazard at the factory that was identified through some sensor on a pipe. Unfortunately, the exact nature and location of the hazard on the pipe are not known. So, Bob decides to remotely inspect the factory before his boss and local public authorities arrive to check.
2. He puts on his head-mounted display on which a cockpit environment is displayed from the mobile metaverse server: a virtual control panel appears in front of him. He can see his hands and the control panel in the cockpit. Bob's application is connected to the mobile metaverse server which enables him to use the service.
3. Bob can tell the mobile metaverse server to configure which surrounding information from the digital twin he wants to monitor. He decides to focus on the 3D representation of the pipe and get real-time sensor information from it, as well as live data from the ambient temperature and gas sensors. The mobile metaverse media displays additional predicted data that temperature is growing, gas concentration is increasing, and that there is a high risk of explosion in less than 10min if this continues. This surrounding information is integrated with other display elements in the cockpit, but he can anchor it in his FOV.
4. While driving along the seaport by remotely controlling the lifting device via its digital twin in the metaverse server, Bob can also see the (hidden) content of other pipes.

Thanks to the 5G mobile metaverse "Tele-operated Driving" service, Bob has been able to drive the vehicle remotely in a reactive way avoiding dangers and finding the leak with the help of the information provided via the digital twins.

The use case related to traffic flow simulation in clause 5.2 already provides requirements and KPIs related to the operation of a moving UE, similar to an AGV. However, that use case does not envision the use of remote control, e.g., using haptic devices and HMD, which trigger new requirements. The use case related to critical healthcare services in clause 5.10 captures the usage of HMD and haptic devices with related requirements and KPIs, which can be generalized to industrial operations. However, this use case does not consider time-critical decisions based on surrounding moving objects in an open area. Neither it relies on real-time digital twin updates to track the characteristics of the environment (e.g., information about pipe content, etc.)

[PR 5.20.6-1] [PR 5.20.6-2] [PR 5.20.6-3] [PR 5.20.6-4]