Tech-invite  3GPPspace  IETFspace
21  22  23  24  25  26  27  28  29  31  32  33  34  35  36  37  38  4‑5x
Top   in Index   Prev   Next

TR 28.908
Study on Artificial Intelligence/Machine Learning (AI/ ML) management

3GPP‑Page   ETSI‑search  
V18.0.0 (PDF)  2023/09  96 p.
Rapporteur:
Mr. Yao, Yizhi
Intel Corporation (UK) Ltd

full Table of Contents for  TR 28.908  Word version:  18.0.0

Here   Top
1 Scope  2 References  3 Definitions of terms, symbols and abbreviations  4 Concepts and overview  5 Use cases, potential requirements and possible solutions  6 Deployment scenarios  7 Conclusions and recommendations  A UML source codes  $ Change history 

 

1  Scope  p. 10

The present document studies the Artificial Intelligence / Machine Learning (AI/ML) management capabilities and services for 5GS where AI/ML is used, including management and orchestration (e.g. MDA, see TS 28.104), 5GC (e.g. NWDAF, see TS 23.288, and NG-RAN (e.g. RAN intelligence defined in TS 38.300 and TS 38.401).
Up

2  References  p. 10

3  Definitions of terms, symbols and abbreviations  p. 11

3.1  Terms  p. 11

3.2  Symbols  p. 11

3.3  Abbreviations  p. 11

4  Concepts and overview  p. 11

4.1  Concepts and terminologies  p. 11

4.2  Overview  p. 11

4.3  AI/ML workflow for 5GS  p. 12

4.3.1  AI/ML operational workflow  p. 12

4.3.2  AI/ML management capabilities  p. 13

5  Use cases, potential requirements and possible solutions  p. 14

5.1  Management Capabilities for ML training phase  p. 14

5.1.1  Event data for ML training  p. 14

5.1.1.1  Description  p. 14

5.1.1.2  Use cases  p. 14

5.1.1.2.1  Pre-processed event data for ML training  p. 14

5.1.1.3  Potential requirements  p. 15

5.1.1.4  Possible solutions  p. 15

5.1.1.5  Evaluation  p. 17

5.1.2  ML entity validation  p. 17

5.1.2.1  Description  p. 17

5.1.2.2  Use cases  p. 17

5.1.2.2.1  ML entity validation performance reporting  p. 17

5.1.2.3  Potential requirements  p. 17

5.1.2.4  Possible solutions  p. 17

5.1.2.4.1  Validation performance reporting by enhancing the existing IOC  p. 17

5.1.2.5  Evaluation  p. 18

5.1.3  ML entity testing  p. 18

5.1.3.1  Description  p. 18

5.1.3.2  Use cases  p. 18

5.1.3.2.1  Consumer-requested ML entity testing  p. 18
5.1.3.2.2  Control of ML entity testing  p. 19
5.1.3.2.3  Multiple ML entities joint testing  p. 19
5.1.3.2.4  Model evaluation for ML testing  p. 19

5.1.3.3  Potential requirements  p. 19

5.1.3.4  Possible solutions  p. 20

5.1.3.4.1  NRM based solution  p. 20

5.1.3.5  Evaluation  p. 21

5.1.4  ML entity re-training  p. 22

5.1.4.1  Description  p. 22

5.1.4.2  Use cases  p. 22

5.1.4.2.1  Producer-initiated threshold-based ML entity re-training  p. 22
5.1.4.2.2  Efficient ML entity re-training  p. 22

5.1.4.3  Potential requirements  p. 22

5.1.4.4  Possible solutions  p. 23

5.1.4.4.1  Producer Initiated Retraining  p. 23
5.1.4.4.2  Efficient ML entity re-training  p. 23

5.1.4.5  Evaluation  p. 24

5.1.5  ML entity joint training  p. 24

5.1.5.1  Description  p. 24

5.1.5.2  Use cases  p. 24

5.1.5.2.1  Support for ML entity modularity - joint training of ML entities  p. 24

5.1.5.3  Potential requirements  p. 25

5.1.5.4  Possible solutions  p. 25

5.1.5.4.1  Support for ML entity modularity - joint training of ML entities  p. 25

5.1.5.5  Evaluation  p. 25

5.1.6  Training data effectiveness reporting and analytics  p. 26

5.1.6.1  Description  p. 26

5.1.6.2  Use cases  p. 26

5.1.6.2.1  Training data effectiveness reporting  p. 26
5.1.6.2.2  Training data effectiveness analytics  p. 26
5.1.6.2.3  Measurement data correlation analytics for ML training  p. 26

5.1.6.3  Potential requirements  p. 27

5.1.6.4  Possible solutions  p. 27

5.1.6.4.1  Possible solution for training data effectiveness reporting  p. 27
5.1.6.4.2  Possible solution for training data effectiveness analytics  p. 28
5.1.6.4.3  Possible solution for measurement data correlation analytics  p. 28

5.1.6.5  Evaluation  p. 30

5.1.7  ML context  p. 30

5.1.7.1  Description  p. 30

5.1.7.2  Use cases  p. 30

5.1.7.2.1  ML context monitoring and reporting  p. 30
5.1.7.2.2  Mobility of ML Context  p. 31
5.1.7.2.3  Standby mode for ML entity  p. 31

5.1.7.3  Potential requirements  p. 32

5.1.7.4  Possible solutions  p. 32

5.1.7.4.1  MLContext <<datatype>> on MLEntity  p. 32
5.1.7.4.2  Mobility of MLContext  p. 32

5.1.7.5  Evaluation  p. 33

5.1.8  ML entity capability discovery and mapping  p. 33

5.1.8.1  Description  p. 33

5.1.8.2  Use cases  p. 34

5.1.8.2.1  Identifying capabilities of ML entities  p. 34
5.1.8.2.2  Mapping of the capabilities of ML entities  p. 34

5.1.8.3  Potential requirements  p. 35

5.1.8.4  Possible solutions  p. 35

5.1.8.5  Evaluation  p. 36

5.1.9  AI/ML update management  p. 36

5.1.9.1  Description  p. 36

5.1.9.2  Use cases  p. 36

5.1.9.2.1  ML entities updating initiated by producer  p. 36

5.1.9.3  Potential requirements  p. 37

5.1.9.4  Possible solutions  p. 37

5.1.9.5  Evaluation  p. 37

5.1.10  Performance evaluation for ML training  p. 37

5.1.10.1  Description  p. 37

5.1.10.2  Use cases  p. 37

5.1.10.2.1  Performance indicator selection for ML model training  p. 37
5.1.10.2.2  Monitoring and control of AI/ML behavior  p. 38
5.1.10.2.3  ML entity performance indicators query and selection for ML training/testing  p. 38
5.1.10.2.4  ML entity performance indicators selection based on MnS consumer policy for ML training/testing  p. 39

5.1.10.3  Potential requirements  p. 39

5.1.10.4  Possible solutions  p. 39

5.1.10.4.1  Possible solutions for performance indicator selection for ML model training  p. 39
5.1.10.4.2  Possible solutions for monitoring and control of AI/ML behavior  p. 40
5.1.10.4.3  Possible solutions for ML entity performance indicators query and selection  p. 41
5.1.10.4.4  Possible solutions for policy-based performance indicator selection  p. 41

5.1.10.5  Evaluation  p. 42

5.1.11  Configuration management for ML training phase  p. 42

5.1.11.1  Description  p. 42

5.1.11.2  Use cases  p. 42

5.1.11.2.1  Control of producer-initiated ML training  p. 42

5.1.11.3  Potential requirements  p. 42

5.1.11.4  Possible solutions  p. 43

5.1.11.4.1  ML training policy configuration  p. 43
5.1.11.4.2  ML training activation and deactivation  p. 43
5.1.11.4.2.1  General framework for activation and deactivation  p. 43
5.1.11.4.2.2  Instant activation and deactivation  p. 43
5.1.11.4.2.3  Schedule based activation and deactivation  p. 43

5.1.11.5  Evaluation  p. 43

5.1.12  ML Knowledge Transfer Learning  p. 44

5.1.12.1  Description  p. 44

5.1.12.2  Use cases  p. 44

5.1.12.2.1  Discovering sharable Knowledge  p. 44
5.1.12.2.2  Knowledge sharing and transfer learning  p. 45

5.1.12.3  Potential requirements  p. 47

5.1.12.4  Possible solutions  p. 47

5.1.12.5  Evaluation  p. 48

5.2  Management Capabilities for AI/ML inference phase  p. 48

5.2.1  AI/ML Inference History  p. 48

5.2.1.1  Description  p. 48

5.2.1.2  Use cases  p. 48

5.2.1.2.1  Tracking AI/ML inference decision and context  p. 48

5.2.1.3  Potential requirements  p. 49

5.2.1.4  Possible solutions  p. 49

5.2.1.5  Evaluation  p. 50

5.2.2  Orchestrating AI/ML Inference  p. 50

5.2.2.1  Description  p. 50

5.2.2.2  Use cases  p. 50

5.2.2.2.1  Knowledge sharing on executed actions  p. 50
5.2.2.2.2  Knowledge sharing on impacts of executed actions  p. 50
5.2.2.2.3  Abstract information on impacts of executed actions  p. 51
5.2.2.2.4  Triggering execution of AI/ML inference functions or ML entities  p. 52
5.2.2.2.5  Orchestrating decisions of AI/ML inference functions or ML entities  p. 52

5.2.2.3  Potential requirements  p. 52

5.2.2.4  Possible solutions  p. 53

5.2.2.5  Evaluation  p. 58

5.2.3  Coordination between the ML capabilities  p. 59

5.2.3.1  Description  p. 59

5.2.3.2  Use cases  p. 59

5.2.3.2.1 

5.2.3.3  Potential requirements  p. 59

5.2.3.4  Possible solutions  p. 60

5.2.3.4.1  Possible solution #1  p. 60

5.2.3.5  Evaluation  p. 60

5.2.4  ML entity loading  p. 60

5.2.4.1  Description  p. 60

5.2.4.2  Use cases  p. 61

5.2.4.2.1  ML entity loading control and monitoring  p. 61

5.2.4.3  Potential requirements  p. 61

5.2.4.4  Possible solutions  p. 62

5.2.4.4.1  NRM based solution  p. 62

5.2.4.5  Evaluation  p. 63

5.2.5  ML inference emulation  p. 63

5.2.5.1  Description  p. 63

5.2.5.2  Use cases  p. 64

5.2.5.2.1  AI/ML inference emulation  p. 64
5.2.5.2.2  Managing ML inference emulation  p. 64

5.2.5.3  Potential requirements  p. 64

5.2.5.4  Possible solutions  p. 65

5.2.5.5  Evaluation  p. 66

5.2.6  Performance evaluation for AI/ML inference  p. 67

5.2.6.1  Description  p. 67

5.2.6.2  Use cases  p. 67

5.2.6.2.1  AI/ML performance evaluation in inference phase  p. 67
5.2.6.2.2  ML entity performance indicators query and selection for AI/ML inference  p. 67
5.2.6.2.3  ML entity performance indicators selection based on MnS consumer policy for AI/ML inference  p. 68
5.2.6.2.4  AI/ML abstract performance  p. 68

5.2.6.3  Potential requirements  p. 68

5.2.6.4  Possible solutions  p. 69

5.2.6.4.1  Possible solutions for AI/ML performance evaluation in inference phase  p. 69
5.2.6.4.2  Possible solutions for ML entity performance indicators query and selection for AI/ML inference  p. 70
5.2.6.4.3  Possible solutions for policy-based performance indicator selection based on MnS consumer policy for AI/ML inference  p. 70
5.2.6.4.4  Possible solutions for AI/ML performance abstraction  p. 70

5.2.6.5  Evaluation  p. 71

5.2.7  Configuration management for AI/ML inference phase  p. 72

5.2.7.1  Description  p. 72

5.2.7.2  Use cases  p. 72

5.2.7.2.1  ML entity configuration for RAN domain ES initiated by consumer  p. 72
5.2.7.2.2  ML entity configuration for RAN domain ES initiated by producer  p. 73
5.2.7.2.3  Partial activation of AI/ML inference capabilities  p. 73
5.2.7.2.4  Configuration for AI/ML inference initiated by MnS consumer  p. 74
5.2.7.2.5  Configuration for AI/ML inference selected by producer  p. 74
5.2.7.2.6  Enabling policy-based activation of AI/ML capabilities  p. 74

5.2.7.3  Potential requirements  p. 74

5.2.7.4  Possible solutions  p. 75

5.2.7.4.1  AI/ML inference function configuration  p. 75
5.2.7.4.1.1  Configuration for AI/ML inference initiated by MnS consumer  p. 75
5.2.7.4.1.2  Configuration for AI/ML inference selected by producer - Context-specific configuration  p. 75
5.2.7.4.2  AI/ML activation  p. 76
5.2.7.4.2.1  General framework for activation and deactivation  p. 76
5.2.7.4.2.2  Instant activation and deactivation  p. 76
5.2.7.4.2.3  Policy based activation and deactivation  p. 76
5.2.7.4.2.4  Schedule based activation and deactivation  p. 76
5.2.7.4.2.5  Gradual activation and deactivation  p. 77

5.2.7.5  Evaluation  p. 78

5.2.8  AI/ML update control  p. 78

5.2.8.1  Description  p. 78

5.2.8.2  Use cases  p. 78

5.2.8.2.1  Availability of new capabilities or ML entities  p. 78
5.2.8.2.2  Triggering ML entity update  p. 78

5.2.8.3  Potential requirements  p. 79

5.2.8.4  Possible solutions  p. 79

5.2.8.5  Evaluation  p. 80

5.3  Common management capabilities for ML training and AI/ML inference phase  p. 80

5.3.1  Trustworthy Machine Learning  p. 80

5.3.1.1  Description  p. 80

5.3.1.2  Use cases  p. 81

5.3.1.2.1  AI/ML trustworthiness indicators  p. 81
5.3.1.2.2  AI/ML data trustworthiness  p. 81
5.3.1.2.3  ML training trustworthiness  p. 82
5.3.1.2.4  AI/ML inference trustworthiness  p. 82
5.3.1.2.5  Assessment of AI/ML trustworthiness  p. 82

5.3.1.3  Potential requirements  p. 83

5.3.1.4  Possible solutions  p. 84

5.3.1.4.1  ML trustworthiness indicators  p. 84
5.3.1.4.2  AI/ML data trustworthiness  p. 85
5.3.1.4.3  ML training trustworthiness  p. 86
5.3.1.4.4  AI/ML inference trustworthiness  p. 86
5.3.1.4.5  Assessment of AI/ML trustworthiness  p. 87

5.3.1.5  Evaluation  p. 87

6  Deployment scenarios  p. 88

7  Conclusions and recommendations  p. 91

A  UML source codes  p. 92

$  Change history  p. 94

Up   Top