Content for  TR 22.874  Word version:  18.2.0

Smartphone is the #1 device that people carry and use everywhere for audio and video recording. It also becomes the first device to exchange media content with friends and family, to publish on social media. High end smartphones embed more and more powerful CPU and GPU and even dedicated AI hardware accelerators. As camera and picture/video quality become a differentiator among high end smartphones, AI/ML models to enhance photo shoots locally emerge on these high-end devices. AI accelerators are expected to enable the execution of complex AI/ML models directly on end-users connected devices; not only photo enhancements but high-quality audio and video content analysis and enhancement are expected to be executed locally on smartphones. Smartphones will consequently become a device to edit media content prior to sharing over the network. With the advent of 5G, new services relying on on-demand downloads of large AI/ML models to be executed in (near) real time on end user device will emerge; depending on the service, the environment, the user's preference, the device characteristics, etc., these DNN models will need to be adapted or updated under strict latency constraints which prevent all of them to be stored locally in advance. Media content analysis combines tasks as object detection, segmentation, face recognition, people counting, human activity tracking. In Table 6.2.1-1, examples of commonly used DNN models for object detection (with their respective sizes) are listed.

Media content edition combines tasks as audio and video quality improvement, language translation, face anonymisation. In Table 6.2.1-2 and Table 6.2.1-3, examples of commonly used DNN models for image super-resolution and for video super-resolution are listed.

Two settings are considered for the use case:

1. Independent user: a person takes a video or starts a video call on its UE in a noisy environment, with difficult lighting conditions, and automatic tagging of scene and objects are embedded in the video.
2. Crowd: During a large event, like a live concert, several thousand people use their UEs to film or photograph the musician band at the same time, and request additional information on the concert like band discography, lyrics, artist facial recognition, instrument/equipment brand. In this context, UEs request the downloads of DNN models to improve the capture and recording of the concert, and to provide information requested by people attending the concert. Several DNN models can be requested by each UE to e.g. execute the following tasks: image shooting and video optimization, artist face recognition, musical instrument identification, audio improvement and lyrics generation. Given the heterogeneous fleet of UEs, thousands of DNN models required by the application/service - can be requested for download. These DNN models are adapted or updated to the UEs operating system type and version, hardware characteristics and environment.

The setting for this use case is as follows. Alice is attending a crowded live concert. She is eager to get movie clips and pictures as a great souvenir of the concert, but she is worried about difficult conditions to get this great souvenir as the conditions for light and sound are very variable, with limited light for the spectators and too much light on the scene. The audio stereo is variable and not well-balanced dependent on where she is among the audience and has a very noisy background. Alice would like to store good quality movie clips and pictures on her private account on the internet for the future, and also post photos and videos tagged with artist name and other relevant information during the concert. As Alice is also an amateur musician, she also wants to get detailed real-time information about the structure of the song, the lyrics and the instruments. The pre-conditions are:

1. Alice is attending a crowded live concert.
2. Her UE is registered to the 5G network.
3. Applications of Alice's smartphone can rely on fine-tuned machine learning models that are available via the network that covers the concert hall. These models are:
   1. An ML model improving photo capture for this concert hall (a model specially fine-tuned for this concert place).
   2. An ML model improving audio capture for this concert hall (a model specially fine-tuned for this concert place).
   3. An ML model specialized in the discography and the lyrics of the band.
   4. An ML model specialized in the artists face recognition.
   5. An ML model specialized in music instrument identification.

1. Shortly after the start of the concert, Alice, as most of the fans, launches the camera application on her mobile phone to film the scene and to get additional information about the band, the individual artists or the songs or instruments.
2. She points her device's camera towards the scene.
3. The environment is very dark with strong light spots. To be fully functional and to render the best user experience, the camera application downloads ad-hoc ML models.
4. The proposed ML models are very performant in this environment but also very heavy in size.
5. The ML models can be used for the whole capture especially if environment remains stable. If the environment changes substantially, or better ML models become available, ML models can be progressively updated accordingly with respect of some operating rules (like a maximum number of ML update per second). In all cases, the camera application continues working seamlessly.
6. The audio and video streams are captured, improved in quality, processed to extract and display additional information, and either stored in real-time on the mobile phone itself or provided as a live stream.

Alice can see that even in harsh light conditions and with the noisy background the photos and videos are great, additional information is provided and all is correctly tagged as requested. The post-conditions are:

1. Photos and videos are either stored on the mobile phone in the improved high quality, ready to be uploaded and shared on social media, or directly shared on social media.
2. Audio recording is high quality with ambient noise reduction, improved stereo balance.
3. Additional information about band, song/lyrics, instruments, etc. are displayed on the mobile phone and stored in media recordings' metadata.
4. Alice can visualize additional information and upload the photos and videos on her social network(s) with the associated tags and information provided by the models, and also store the above on her personal media server.

The performance requirements for high data rate and traffic density scenarios are found in clause 7.1 of TS 22.261. The scenario Broadband access in a crowd is relevant for the use case of very dense crowds, for example at stadiums or concerts. In addition to a very high connection density, the users can share their experience, i.e. what they see and hear. This can put a higher requirement on the uplink than the downlink. This new use case has some requirements on the downlink not covered by the existing requirements, see Table 6.2.5-1.