Tech-invite 3GPPspace IETFspace

21 22 23 24 25 26 27 28 29 31 32 33 34 35 36 37 38 4‑5x

Content for TR 26.997 Word version: 18.0.0

1… 4…

4 General p. 10

4.1 Overview of the IVAS Codec Work Item p. 10

4.2 Presentation of the Following clauses p. 13

5 Terms of Reference p. 13

6 Selection Process p. 15

7 Introduction to the Testing of the IVAS codec p. 16

7.1 IVAS Selection Phase Testing p. 16

7.1.1 General methodology p. 16

7.1.1.1 Overview p. 16

7.1.1.2 P.800 p. 16

7.1.1.3 BS.1534 p. 16

7.1.2 Opinion Scales p. 17

7.1.2.1 ITU-T P.800 DCR p. 17

7.1.2.2 ITU-R BS.1534 p. 18

7.1.3 Material p. 18

7.1.3.1 Overview p. 18

7.1.3.2 Speech Material for P.800 testing p. 19

7.1.3.3 Background Material p. 19

7.1.3.4 Music and Mixed Content Material for P.800 testing p. 19

7.1.3.5 Critical Generic Audio Items for BS.1534 testing p. 19

7.1.3.5.1 Steps of Critical Test Item Selection p. 19

7.1.3.5.2 Test Material p. 20

7.1.3.5.3 Training material p. 20

7.1.4 Listening Systems and Listening Environments p. 21

7.1.5 Experimental Procedure for P.800 experiments p. 21

7.1.6 Subjective Experiments p. 21

8 Important Notes about the Interpretation of Test Results p. 24

8.1 P.800 Testing p. 24

8.2 BS.1534 Testing p. 24

8.3 Analysis of Selection Phase Results p. 25

9 IVAS Performances p. 26

9.1 Mono p. 26

9.2 Stereo p. 26

9.2.1 Overview p. 26

9.2.2 Selection Experiment P800-1 (Stereo/Binaural, Clean Speech, Headphone Presentation) p. 27

9.2.3 Selection Experiment P800-2 (Stereo, Speech+Background, Headphone Presentation) p. 28

9.2.4 Selection Experiment P800-3 (Stereo, Mixed and Music, Headphone Presentation) p. 30

9.2.5 Selection Experiment BS1534-1a (Stereo, Generic Audio, 48 and 64 kbps, Headphone Presentation) p. 32

9.2.6 Selection Experiment BS1534-1b (Stereo, Generic Audio, 96 and 128 kbps, Headphone Presentation) p. 34

9.3 Scene-based Audio (SBA, Ambisonics) p. 35

9.3.1 Overview p. 35

9.3.2 Selection Experiment P800-4 (FOA, Clean Speech, Headphone Presentation) p. 35

9.3.3 Selection Experiment P800-5 (FOA, Speech+Background, Headphone Presentation) p. 37

9.3.4 Selection Experiment BS1534-4a (FOA, Generic Audio, 96, 128 and 160 kbps, Headphone Presentation) p. 39

9.3.5 Selection Experiment BS1534-4b (HOA2, Generic Audio, 160 and 192 kbps, Headphone Presentation) p. 41

9.3.6 Selection Experiment BS1534-5a (HOA3, Generic Audio, 192 and 256 kbps, Headphone Presentation) p. 42

9.3.7 Selection Experiment BS1534-5b (HOA3, Generic Audio, 384 and 512 kbps, 7.1+4 Loudspeaker Presentation) p. 44

9.4 Objects (Independent Streams with Metadata, ISM) p. 45

9.4.1 Overview p. 45

9.4.2 Selection Experiment P800-6 (1 Object, Clean speech, Headphone Presentation) p. 45

9.4.3 Selection Experiment P800-7 (2 Objects, Clean speech, Headphone Presentation) p. 47

9.4.4 Selection Experiment BS1534-6a (3 Objects, Generic Audio, 48, 64 and 96 kbps, Headphone Presentation) p. 49

9.4.5 Selection Experiment BS1534-6b (4 Objects, Generic Audio, 96, 128 and 256 kbps, Headphone Presentation) p. 51

9.5 Metadata-assisted Spatial Audio (MASA) p. 52

9.5.1 Overview p. 52

9.5.2 Selection Experiment P800-8 (MASA, Clean speech, Headphone Presentation) p. 52

9.5.3 Selection Experiment P800-9 (MASA, Speech+Background, Headphone Presentation) p. 54

9.5.4 Selection Experiment BS1534-7a (MASA, Generic Audio, 96 and 128 kbps, Headphone Presentation) p. 56

9.5.5 Selection Experiment BS1534-7b (MASA, Generic Audio, 192 and 256 kbps, Headphone Presentation) p. 58

9.6 Multi-Channel (MC) p. 59

9.6.1 Overview p. 59

9.6.2 Selection Experiment BS1534-2a (MC 5.1, Generic Audio, 64 and 96 kbps, 5.1 Loudspeaker Presentation) p. 59

9.6.3 Selection Experiment BS1534-2b (MC 5.1, Generic Audio, 128 and 160 kbps, 5.1 Loudspeaker Presentation) p. 61

9.6.4 Selection Experiment BS1534-3a (MC 7.1.4, Generic Audio, 128 and 160 kbps, 7.1+4 Loudspeaker Presentation) p. 62

9.6.5 Selection Experiment BS1534-3b (MC 7.1.4, Generic Audio, 384 and 512 kbps, 7.1+4 Loudspeaker Presentation) p. 64

9.7 Combined Objects and SBA (OSBA) p. 65

9.7.1 Overview p. 65

9.7.2 Optional additional information provided for codec selection p. 65

9.8 Combined Objects and MASA (OMASA) p. 67

9.8.1 Overview p. 67

9.8.2 Optional additional information provided for codec selection p. 67

9.9 Stereo operation with EVS compatible mono downmix stream p. 67

9.9.1 Overview p. 67

9.9.2 Optional additional information provided for codec selection p. 68

9.10 Rendering p. 69

9.11 Split Rendering p. 69

9.11.1 General p. 69

9.11.2 Overview p. 69

9.11.3 ISAR Selection Phase Requirements and Objectives p. 69

9.11.4 ISAR Selection Experiment ISAR_BS1534-1 (HOA3, Generic Audio, 768 kbps Split Rendering Link, Headphone Presentation) p. 70

9.11.4.1 Overview p. 70

9.11.4.2 Test conditions p. 70

9.11.4.3 Result plots p. 70

9.11.4.4 Statistical analysis p. 72

9.11.4.5 Experimental conclusions p. 72

9.11.5 ISAR Selection Experiment ISAR_BS1534-2 (MC 7.1+4, Generic Audio, 768 kbps Split Rendering Link, Headphone Presentation) p. 72

9.11.5.1 Overview p. 72

9.11.5.2 Test conditions p. 72

9.11.5.3 Result plots p. 72

9.11.5.4 Statistical analysis p. 74

9.11.5.5 Experimental conclusions p. 74

9.11.6 ISAR Selection Experiment ISAR_BS1534-3 (4 Objects, Generic Audio, 768 kbps Split Rendering Link, Headphone Presentation) p. 74

9.11.6.1 Overview p. 74

9.11.6.2 Test conditions p. 74

9.11.6.3 Result plots p. 74

9.11.6.4 Statistical analysis p. 76

9.11.6.5 Experimental conclusions p. 76

9.11.7 ISAR Selection Experiment ISAR_BS1534-4 (MASA 2TC, Generic Audio, 768 kbps Split Rendering Link, Headphone Presentation) p. 76

9.11.7.1 Overview p. 76

9.11.7.2 Test conditions p. 76

9.11.7.3 Result plots p. 76

9.11.7.4 Statistical analysis p. 78

9.11.7.5 Experimental conclusions p. 78

10 Objective Evaluations p. 78

10.1 Complexity and Delay Analysis p. 78

10.1.1 Complexity p. 78

10.1.2 Algorithmic Delay p. 78

A ToR Tests in Selection Phase p. 80

A.1 ToR Tests for Requirements p. 80

B Overall Characterization of the IVAS Codec p. 82

C IVAS Selection Phase Testplan p. 84

C.1 Experiment P800-1: Stereo/Binaural Clean Speech Test p. 84

C.1.1 Experiment setup p. 84

C.1.2 Content type categories and scene definitions p. 86

C.2 Experiment P800-2: Stereo Speech and Background Test p. 88

C.2.1 Experiment setup p. 88

C.2.2 Content type categories and scene definitions p. 89

C.3 Experiment P800-3: Stereo Mixed and Music Test p. 92

C.3.1 Experiment setup p. 92

C.4 Experiment P800-4: FOA Clean Speech Test p. 94

C.4.1 Experiment setup p. 94

C.4.2 Content type categories and scene definitions p. 95

C.5 Experiment P800-5: FOA Speech+background Test p. 97

C.5.1 Experiment setup p. 97

C.5.2 Content type categories and scene definitions p. 98

C.6 Experiment P800-6: 1-Object Clean Speech Test p. 100

C.6.1 Experiment setup p. 100

C.6.2 Content type categories and scene definitions p. 102

C.7 Experiment P800-7: 2-Objects Clean Speech Test p. 103

C.7.1 Experiment setup p. 103

C.7.2 Content type categories and scene definitions p. 106

C.8 Experiment P800-8: MASA Clean Speech Test p. 107

C.8.1 Experiment setup p. 107

C.8.2 Content type categories and scene definitions (Exp P800-8: Clean speech) p. 110

C.9 Experiment P800-9: MASA Speech+Background Test p. 111

C.9.1 Experiment setup p. 111

C.9.2 Content type categories and scene definitions (Exp P800-9: Speech+Background) p. 112

C.10 Experiment BS1534-1a: Stereo p. 114

C.11 Experiment BS1534-1b: Stereo p. 115

C.12 Experiment BS1534-2a: Multi-channel 5.1 p. 117

C.13 Experiment BS1534-2b: Multi-channel 5.1 p. 119

C.14 Experiment BS1534-3a: Multi-channel 7.1+4 p. 120

C.15 Experiment BS1534-3b: Multi-channel 7.1+4 p. 122

C.16 Experiment BS1534-4a: FOA p. 124

C.17 Experiment BS1534-4b: HOA2 p. 126

C.18 Experiment BS1534-5a: HOA3 p. 128

C.19 Experiment BS1534-5b: HOA3 p. 129

C.20 Experiment BS1534-6a: Objects p. 130

C.21 Experiment BS1534-6b: Objects p. 131

C.22 Experiment BS1534-7a: MASA p. 133

C.23 Experiment BS1534-7b: MASA p. 134

D IVAS Characterization Phase Testplan p. 136

E Relevant Permanent Documents p. 137

E.1 Overview p. 137

E.2 IVAS-3: IVAS Performance Requirements p. 137

E.3 IVAS-4: IVAS Design Constraints p. 137

E.4 IVAS-5: IVAS Selection Rules p. 137

E.5 IVAS-6: IVAS Selection Deliverables p. 137

E.6 IVAS-7a: Processing Plan for Selection Phase p. 137

E.7 IVAS-7b: Processing Plan for Characterization Phase p. 137

E.8 IVAS-8a: Test Plan for Selection Phase p. 137

E.9 IVAS-8b: Test Plan for Characterization Phase p. 137

E.10 ISAR Permanent Document: Testing Aspects for Phase/Track 2/a p. 138

F Subjective Test Raw Data and Analyses p. 139

F.1 IVAS Selection Phase: Global Analysis Laboratory report p. 139

$ Change history p. 140