Content for TR 26.843 Word version: 16.0.0

3.1 Definitions 3.2 Abbreviations 4.1 Introduction
...

0 Introduction p. 5

The EVS coder (TS 26.441) provides enhanced quality for speech and audio communications compared to AMR-WB and 3GPP has standardized both a fixed-point version (TS 26.442) and a floating-point version (TS 26.443). Currently in TS 26.444 (Codec for Enhanced Voice Services (EVS); Test sequences) the conformance of the EVS coder implementation is achieved by checking the bit-exactness of output test vectors with the reference test vectors for both encoder and decoder, for both the fixed-point and floating-point implementations.

However, the bit-exact criteria defined in TS 26.444 is of very limited use for the floating-point implementation in TS 26.443, as the output values will change slightly without affecting the speech/audio quality - depending on the compiler, compile options, OS and platform - and therefore failing the bit-exactness test. This has the effect that the EVS floating-point code cannot generally be used for 3GPP voice services as the test vectors have been generated using Microsoft Visual Studio version 10 which is unlikely to match the target platform.

The product and application space using voice services is changing, resulting in different architectures using a variety of different types of core processing units. Being able to use either fixed-point or floating-point embedded implementation based on architectural capabilities would allow a wider and faster proliferation of EVS, thereby benefiting end user experience. In addition, it would provide more flexibility in architectural implementations regarding factors such as power and cost.

The present document investigates possible tools and criteria to develop non bit-exact conformance for the floating-point code in TS 26.443, ensuring that high quality floating-point implementations preserve the quality of EVS.

1 Scope p. 6

The present document provides a study on the Conformance of Non Bit Exact implementation for EVS floating point standard in TS 26.443. The study focuses on:

To investigate the behaviour of different implementations of the floating-point reference code (TS 26.443), for example, those built with different versions / settings of various compilers and running on various floating-point architectures.
To do the investigation using different test material, including clean speech, noisy speech, mixed/music content and taking into account interoperability aspects including floating-point - fixed-point and among various floating-point implementations.
To identify and propose reliable conformance criteria and methodologies that would be able to reject any undesirable deviation, i.e. bad implementation.
To develop one or more tools, in the form of scripts or executables, that could be used for determining acceptance/rejection based on the provided conformance criteria.
To develop any potential additional test vectors that would be needed.
To propose recommendation(s) on the suitability of the potential new non bit-exact conformance process for 3GPP services (e.g., gives carrier-grade quality).

2 References p. 6

The following documents contain provisions which, through reference in this text, constitute provisions of the present document.

References are either specific (identified by date of publication, edition number, version number, etc.) or non specific.
For a specific reference, subsequent revisions do not apply.
For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same Release as the present document.

[1]

TR 21.905: "Vocabulary for 3GPP Specifications".

[2]

TS 26.442: "Codec for Enhanced Voice Services (EVS); ANSI C code (fixed point)".

[3]

TS 26.443: "Codec for Enhanced Voice Services (EVS); ANSI C code (floating point)".

[4]

TS 26.444: "Codec for Enhanced Voice Services (EVS); Test Sequences".

[5]

Tdoc S4-141287: "Verification of the EVS Floating Point Code".

[6]

Tdoc S4-141392: "EVS Permanent Document EVS-7c: Processing functions for characterization phase".

[7]

TR 26.952: "Codec for Enhanced Voice Services (EVS); Performance Characterization".

[8]

Proc. IEEE Digital Signal Processing Workshop: "Comparison of distance measures in discrete spectral modelling", B. Wei and J. D. Gibson, Oct. 2000.

[9]

ITU-T Recommendation P.863.1 (09/2014): "Application guide for recommendation ITU-T P.863".

[10]

ISO/IEC 14496-26:2010: "Information technology -- Coding of audio-visual objects -- Part 26: Audio conformance".

[11]

ITU-R Recommendation BS.1387-1 (11/2001): "Method for objective measurements of perceived audio quality".

[12]

ITU-T Recommendation P.863 (03/2018): "Perceptual objective listening quality assessment".

3 Definitions and abbreviations p. 7

3.1 Definitions p. 7

For the purposes of the present document, the terms and definitions given in TR 21.905 and the following apply. A term defined in the present document takes precedence over the definition of the same term, if any, in TR 21.905.

3.2 Abbreviations p. 7

For the purposes of the present document, the abbreviations given in TR 21.905 and the following apply. An abbreviation defined in the present document takes precedence over the definition of the same abbreviation, if any, in TR 21.905.

Up-link

4 Overview p. 7

4.1 Introduction p. 7

The EVS coder (TS 26.441) provides enhanced quality for speech and audio communications relative to AMR-WB and 3GPP has standardized both a fixed-point version (TS 26.442) and a floating-point version (TS 26.443). Currently in TS 26.444 (Codec for Enhanced Voice Services (EVS); Test sequences) the conformance of the EVS coder implementation is achieved by checking the bit-exactness of output test vectors with the reference test vectors for both encoder and decoder, for both the fixed-point and floating-point implementations.

The bit-exact criteria defined in TS 26.444 are of very limited use for the floating-point implementation in TS 26.443 as the output values will be similar but not bit-exact for different compilers, compile options, OS and platform used and therefore failing the bit-exactness test.

This technical report aims at documenting possible methods for non bit-exact conformance process for floating-point implementation that would allow conforming implementations to be used in all scenarios acceptable for bit-exact implementations of fixed-point version (TS 26.442) and floating-point version (TS 26.443).

The scope of the study item is to assess the use of various floating-point processing cores and compilers with various levels of optimization, and establish potential conformance criteria and a tool(s) that could be used for confirming conformance under those variations. The scope of the intended conformance tool(s) is to assess the conformance of implementations to the developed criteria. The conformance criteria to be developed will aim at accepting proper floating-point processing compute core and compiler optimizations, while rejecting all bad implementations, for example coming from functional or code changes, too aggressive optimization of the compiler or insufficient arithmetic precision.

In Clause 5 various methods are described.

In Clause 6 results obtained with the various methods described in Clause 5 are presented..

In Clause 7 possible conformance process and criteria are discussed.

Interoperability is an important feature of the coder, and implication for non bit-exact conformance will be investigated in Clause 8.

Clause 9 will look at coverage of the proposed method.

Clause 10 will address any other topics relevant to the context of this study.

Clause 11 will conclude on the feasibility of using non bit-exact tools and criteria for EVS floating point conformance.