The SMG2-Speech experts Group (SEG) started its activity early in 1995 for the standardization of an Enhanced Full Rate speech codec. The Group produced a test plan for the first phase of testing (pre-selection phase) which is described in permanent document SEG 4 (ETSI SMG2 SEG: SEG 4 (v 1.0) "A Subjective Pre-Selection Test Plan for the Enhanced Full Rate Speech Coding Algorithm") to assess the performance of the submitted candidates. This test plan is based on the general knowledge coming from past ITU-T and ETSI activities on codec evaluation (GSM half rate and ITU-T 8 kbit/s recent exercises for instance). At the end of this Pre-selection Phase, SMG decided to standardize the PCS 1 900 codec, known as the US 1 codec and no formal characterisation testing has been performed for the selected codec. The present document therefore reports the results from the Pre-selection and Verification Phase of testing only. Consequently, the results reported here are less detailed, and the confidence intervals for them are wider, than those obtained for the GSM half rate standardization (GSM 06.08, [3]) where specific and detailed characterisation testing was performed. In addition, not all laboratories followed the same pre-selection test plan, further complicating the interpretation of the results. The following experiments included in SEG 4 were carried out by several laboratories in the Pre-selection Phase:

• Experiment 1: Quality under error and tandeming conditions (A-law, Modified IRS);
• Experiment 2: Quality under background noise conditions (Vehicular noise, UPCM, NoIRS);
• Experiment 3: Quality under background noise conditions (Background music, UPCM, NoIRS);
• Experiment 4: Talker Dependency (UPCM, NoIRS);
• Experiment 5: Quality under high error conditions -EP3 (A-law, Modified IRS).

A practical 'indirect' method of performance comparison between different results was adopted utilising the Modulated Noise Reference Unit (MNRU) (see note) as a reference degradation. The MNRU provides the additional function of allowing normalisation of results across different laboratories carrying out the same experiment, through the conversion of MOS scores to Equivalent Q (dB). The Q (dB) values introduced in a test normally range from 0 to 50 dB. In SEG 4, both Experiment#1 and Experiment#5 on error conditions covers this range, the other experiments do not. Only four laboratories ran tests which followed the Pre-selection Test Plan described in SEG 4 (BT/lab1, CNET/lab2, Tele Denmark/lab3, NEC/lab4). MOTOROLA/lab5 participated in the Pre-selection Phase but their experiments did not comply with SEG 4. TI/lab8 ran one experiment only from SEG 4. Results produced by COMSAT/lab6 following a NOKIA-designed test plan are part of standardization of the codec in North America and NOKIA/lab7 performed complementary experiments during the ETSI Pre-selection Phase. As no further analysis have been undertaken to allow the averaging of scores across the different laboratories, results are reported in the annex on a laboratory-by-laboratory basis. For error and tandeming conditions, results are reported in terms of Equivalent Q (dB) values. For background noise conditions and talker dependency, results are reported in terms of DMOS values with either Confidence Interval (CI) or Standard Deviation (SD) as there is insufficient data available to normalise across laboratories via MNRU conditions. The quality performance of the EFR codec is compared to High and Low references introduced in permanent documents SEG 3 (ETSI SMG2 SEG: SEG 3 "Selection Criteria for the Enhanced Full Rate Speech Coding Algorithm - Speech Quality Requirements") and SEG 4 (ETSI SMG2 SEG: SEG 4 (v 1.0) "A Subjective Pre-Selection Test Plan for the Enhanced Full Rate Speech Coding Algorithm", Section 7). These references were chosen as representative of the "minimum" and "objective" performance targets respectively, and are reported in Table 1.

  A Figure showing the general trend of the EFR behaviour for error conditions in noise-free environment, compared to the high (G.728) and low (TCH-FS) references is added to individual laboratories' quantitative results (Figure 15). The general quality performance of the EFR codec is summarised in table 15. In the Verification Phase, the behaviour of the EFR codec under the following test conditions was tested:

• behaviour of the DTX System;
• performance with DTMF tones;
• performance with network information tones;
• performance with special input signals;
• performance with music signals;
• performance with noise signals;
• performance with different languages;
• delay of the TCH-EFR;
• frequency response;
• complexity.

The results of these tests are also included in this report under the respective clauses. Furthermore, the EFR codec was checked for correct functioning for the following items:

• test of overload point;
• SID frame encoding;
• muting behaviour;
• idle channel behaviour.

No artefact or malfunctioning was detected for these items.

The present document gives background information on the performance of the GSM enhanced full rate speech codec. Experimental results from the Pre-selection and Verification tests carried out during the standardization process by the SEG (Speech Expert Group) are reported to give a more detailed picture of the behaviour of the GSM enhanced full rate speech codec under different conditions of operation.

The following documents contain provisions which, through reference in this text, constitute provisions of the present document.

• References are either specific (identified by date of publication, edition number, version number, etc.) or non specific.
• For a specific reference, subsequent revisions do not apply.
• For a non-specific reference, the latest version applies. In the case of a reference to a 3GPP document (including a GSM document), a non-specific reference implicitly refers to the latest version of that document in the same Release as the present document.

For the purposes of the present document, the following abbreviations apply: A/D ADPCM ACR BSC BTS C/I CI CNI CRC D/A DAT DCR DSP DTMF DTX EFR ESP FR GBER GSM HR IRS ITU T MNRU Mod. IRS MOPS MOS MS MSC PCM PSTN Q SD SEG SID SMG TCH-EFS TCH-FS TCH-HS TDMA TMOPS UPCM VAD WMOPS   Four different Error Patterns (EP0, EP1, EP2 and EP3) were used, where: EP0 EP1 EP2 EP3