These capabilities whose service requirements are defined in clause 5.1.1.2 enable diverse mobile metaverse services.
One important class of services involves several users who take part in mobile metaverse services simultaneously, for example, to support a 'virtual sport event' where some of the environment or objects in the match are virtual, that is, they are produced by an application that provides the user with XR media. Users could be local (in the same location) or remote and have a service experience that is immersive and meets the expectations set by the interactive activity.
Another important class of services are those that require coordination of diverse service data flows of sensor data and media in order to satisfy the needs of a digital twin or situational awareness service.
The service requirements in this clause correspond to means by which the 5G system provides access to digital assets and communication services for mobile metaverse services so that:
the service experience of users of the same service are compatible and consistent;
the service experience of a user involving media from different services is consistent, even when these services have different performance characteristics;
the services can operate over a sufficient duration for devices with constrained energy storage;
the services can communicate efficiently to a large number of authorized users;
the communication performance for specific mobile metaverse services to specific users can be monitored and exposed to third parties.
Subject to operator policy, the 5G system shall support a mechanism that enables flexible adjustment of communication services based on e.g., the type of devices (e.g., wearables), or communication duration (e.g., more than one hour), such that the services can be operated with reduced energy utilization.
[R-5.1.1-002]
The 5G system shall provide a means to associate and coordinate data flows related to one or multiple UEs e.g., associated with the same object in digital twin applications provided by the mobile metaverse service.
[R-5.1.1-003]
Subject to operator policy, regulatory requirements and user consent, the 5G system (including IMS) shall be able to expose network performance information (e.g., observed or predicted bitrate, latency or packet loss) related to one or more users to an authorized third party metaverse application.
[R-5.1.1-004]
Subject to operator policy, the 5G system (including IMS) shall support a mechanism, including enabling one or more authorized third party(ies) to coordinate multiple service data flows of a single mobile metaverse service delivered to/from one or more UE(s). Multiple UEs may be associated with one user/location or different users at different locations potentially using different access networks, i.e., 3GPP and non-3GPP.
[R-5.1.1-005]
The 5G system shall enable the coordination of diverse media, transmitted to a UE from one or more mobile metaverse services associated with a physical location, to be combined to form a localized service experience.
[R-5.1.1-006]
Subject to operator policy, the 5G system shall support exposure mechanisms enabling an authorized third party to determine one or more subscribers to whom mobile metaverse media can be distributed in a resource efficient manner.
[R-5.1.1-007]
Subject to operator policy and user consent, the 5G system shall support a means to provide resource efficient communication of third party mobile metaverse media to one or more subscribers.
[R-5.1.1-008]
The 5G system shall provide a means to maintain consistent user experience, for a given UE, when XR media from different mobile metaverse services have different communication performance, e.g., resolution, latency or packet loss.
Localized mobile metaverse services are immersive and integrated into a user's ordinary experiences. Such service experiences are location-related and can include presentation of AR, MR media.
Localized experiences are effectively present in the user's environment, so that the mobile metaverse media provided for a given mobile metaverse service is both appropriate to and integrated with both the physical world and with mobile metaverse media content displayed. Localized mobile metaverse services can be associated with specific places (3D locations in the physical world). The association between these places and service information is termed a spatial anchor.
Spatial anchors enable mobile metaverse services to be discovered and accessed, if the user is authorized. For example, the service information can convey the mobile metaverse server access information. When the user's application accesses the mobile metaverse service, the media associated with the service can be obtained by the user.
Spatial anchors can associate diverse information with spatial location, beyond access control and access information of mobile metaverse services. Type of service information can also allow a user to discover appropriate spatial anchors, e.g., when the user seeks restaurants.
Spatial anchors can be defined by third parties, e.g., service providers, to offer relevant localized services, e.g., associated with specific items or features in their place of business. This information and its associated authorization information, determining who can discover the spatial anchor, can be managed - created, deleted and modified.
Users' localization is important in order to discover spatial anchors. The 5G system offers a spatial localization service to determine this information. Using sensor data related to the user's location, the 5G system can identify where the user is. This is possible by means of processing the sensor data as well as a spatial map. The result, precise user location and orientation in space, can be exposed to authorized third parties.
The spatial map is created using processed sensor data. The 5G system supports a spatial mapping service to customers that, for example, want to offer mobile metaverse services associated with spatial anchors on their premises. Creation of a spatial map for a location makes localization there possible, as well as assignment of spatial anchors in that location.
Subject to operator policy, the 5G system shall provide a means to define and expose to an authorized third party a spatial anchor, i.e., an association between a physical location (a point or volume in three-dimensional space) and service information.
[R-5.2.1-002]
Subject to operator policy, the 5G system shall enable an authorized third party to request the information associated with a specific spatial anchor.
[R-5.2.1-003]
Subject to operator policy, regulatory requirements and user consent, the 5G system shall provide a means for a UE to provide sensor data (e.g., from UE sensors, cameras, etc.) to the network in order to derive localization information, e.g., to produce or modify a spatial map or discover or find spatial anchors. The 5G system shall enable an authorized third party to obtain all the spatial anchors in a given three dimensional area.
[R-5.2.1-004]
Subject to operator policy and regulatory requirements, the 5G system shall support mechanisms to expose a spatial map or derived localization information to authorized third parties.
[R-5.2.1-005]
Subject to operator policy, regulatory requirements and user consent, the 5G System shall be able to process and expose information related to a UE's location and direction of orientation in space to authorized third parties.
A user can take part in mobile metaverse services that provide digital representations of several other users simultaneously, for example, to support a Conference using XR media. As these services are interactive and immersive, the 5G system provides a means so that the experience of each user of the same service is compatible and consistent. Users can participate together in this way, whether some of those in the Conference service are located in the same place: these users can experience remote users as AR or MR media. This media is conversational - meaning that all parties can participate, and real time - meaning that all users perceive each other's actions effectively simultaneously.
Avatars are employed in mobile metaverse services mainly to represent a user. To improve the user experience of this representation, the 5G system makes it possible to associate the user and the avatar representation and to expose this information to authorized third parties. This information could be valuable, as avatars can represent organizations, individual users, even groups of users.
The 5G system supports a means by which user's pose, gestures and expressions are captured as input for the conversational mobile metaverse service. Devices (e.g., UEs) can capture this information in a standardized form, which is used for the creation of an animated digital representation of the user (e.g., 3D avatar) that can be presented to other users. The system supports communication of this animated digital representation as "avatar media" as well as audio and other media as needed. The 5G system supports the creation of avatar media. Privacy and user consent is needed for sensor data used to capture a user's pose, gestures and (facial) expressions is sent from the UE to the network encoded as avatar media to be rendered.
The 5G system further supports predictive digital representation models in that it enables presentation of avatar media to users based upon timing and other information, so that information can be extrapolated or inferred even if it is not yet available from the network. Correction of these predictions is possible over time, as new information becomes available, e.g., from the real world. The 5G system can in this way enable the user experience of synchronized mobile metaverse services even where communication latencies (e.g., due to distance between users) do not permit this.
Capabilities of UEs differ. For example: some UEs can render avatar media, others video, others only text. To support interactive avatar communication, media transcoding can occur in the 5G network. Similarly, the display capabilities of UEs also differ. Avatar, video or text data can be transcoded as appropriate to be displayed to the user.
Capabilities of users also vary. To support accessibility for those with disability due to physical (e.g., impaired hearing, sight, etc.), environmental (e.g., in a noisy environment), conversational media can be transcoded. This is consistent with the objective for Total Conversation [5], clause 4.5. Avatar functionality provides new options, as media can be transcoded to and from an avatar call.
The 5G system shall support 5G CN to provide real-time feedback in support of conversational XR communication among multiple users simultaneously.
[R-5.2.2-002]
Subject to operator policy and user consent, the 5G system (including IMS) shall support multimedia conversational communications between two or more users including transfer of real time avatar media and audio media.
[R-5.2.2-003]
Subject to operator policy and user consent, the 5G system (including IMS) shall support change of media types between video and avatar media for parties of a multimedia conversational communication.
[R-5.2.2-004]
Subject to operator policy, the 5G system (including IMS) shall support transcoding between media such as text, video and avatar media in multimedia conversational communications.
[R-5.2.2-005]
Subject to operator policy, regulatory requirements and user consent, the 5G system (including IMS) shall support the capabilities of rendering the avatar based on the body movement information (e.g., body motion or facial expression) of a human user.
[R-5.2.2-006]
Subject to operator policy, regulatory requirements and user consent, the 5G system (including IMS) shall support the encoding of sensor data capturing the facial expression and movement and gestures of a person, in a standard form.
[R-5.2.2-007]
Subject to operator policy and regulatory requirements, the 5G system shall support mechanisms to uniquely identify an avatar, to associate the avatar with a subscriber and to expose this association to authorized third parties.
[R-5.2.2-008]
The 5G system (including IMS) shall support compensating for the end-to-end communication latency between the users and/or objects involved in a multimedia conversational communication during rendering the digital representation (e.g., avatar) of the users and/or objects involved (e.g., by using a predictive digital representation model).
Mobile metaverse services can depend upon information that is associated with the user, e.g., User Identifiers and personal data that are commonly required and represented in a machine-readable format. The requirements as described in clause 26a of TS 22.101 apply for identification of users. These can be used to provide proof for regulatory constrained service, e.g., proof of residential address for services that are restricted to local residents.
Further, the services can benefit from common information, such as avatar parameters and configuration information, so that a user's digital representation is consistent across different applications. Users can benefit from the support of associating their digital assets with different User Identities, to flexibly control their stored information.
Finally, some more specific information used by different services can also be shared in different mobile metaverse services and be considered 'digital assets' in that the user needs or could benefit from having this information available when access mobile metaverse services. The 5G system provides a means to improve interoperability of the use of digital assets between the various metaverse platforms. For this, digital asset formats e.g., avatar format have to be adaptable and consistent with the characteristics and constraints of the service (e.g., performance, graphics). According to regulations, this information can be considered as personal data.
An example of such service is the EU digital wallet initiative [6]. Both the digital wallet and the digital asset management functionality described in the present document emphasize the need for security, privacy and control over access to authorized parties. This initiative is summarized in Annex B.
Subject to operator policy, regulatory requirements and user consent, the 5G system shall be able to provide functionality to store digital assets associated with a user, and to remove such digital assets associated with a user.
[R-5.2.3-002]
Subject to operator policy, regulatory requirements and user consent, the 5G system shall provide a means to allow a user to securely access and update their digital assets.
[R-5.2.3-003]
Subject to operator policy and user consent, the 5G system shall be able to allow an authorized third party to retrieve the digital asset(s) associated with a user, e.g., when the user accesses a specific application.
[R-5.2.3-004]
The 5G system (including IMS) shall support at least one common avatar format to enable interoperability with multiple immersive mobile metaverse services.
[R-5.2.3-005]
The 5G system shall be able to associate a stored digital asset with one or more User Identities.
[R-5.2.3-006]
Subject to operator policy, regulatory requirements and user consent, the 5G system shall support a mechanism for users to define conditions (e.g., based on user location information) to restrict the access to, and management of, stored digital assets associated with User Identity.
[R-5.2.3-007]
The 5G system shall support mechanisms to request specific formats of stored digital assets associated with a user by an authorized mobile metaverse service.