3. Performance Metrics
3.1. Physical Layer-SONET
3.1.1. Pointer Movements
3.1.1.1. Pointer Movement Propagation.
Objective: To determine that the SUT does not propagate pointer movements as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of IP PDUs at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The cell payload SHOULD contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
3) Count the IP PDUs that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test, else lower the test device traffic rate until the counts are the same. 4) Inject one forward payload pointer movement. Verify that the SUT does not change the pointer. 5) Inject one forward payload pointer movement every 1 second. Verify that the SUT does not change the pointer. 6) Discontinue the payload pointer movement. 7) Inject five forward payload pointer movements every 1 second. Verify that the SUT does not change the pointer. 8) Discontinue the payload pointer movement. 9) Inject one backward payload pointer movement. Verify that the SUT does not change the pointer. 10) Inject one backward payload pointer movement every 1 second. Verify that the SUT does not change the pointer. 11) Discontinue the payload pointer movement. 12) Inject five backward payload pointer movements every 1 second. Verify that the SUT does not change the pointer. 13) Discontinue the payload pointer movement. Reporting Format: The results of the pointer movement propagation test SHOULD be reported in a form of a table. The rows SHOULD be labeled single pointer movement, one pointer movement per second, and five pointer movements per second. The columns SHOULD be labeled pointer movement and loss of pointer. The elements of the table SHOULD be either True or False, indicating whether the particular condition was observed for each test. The table MUST also indicate the IP PDU size in octets and traffic rate in IP PDUs per second as generated by the test device.
3.1.1.2. Cell Loss due to Pointer Movement.
Objective: To determine if the SUT will drop cells due to pointer movements as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of cells at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The cell payload SHOULD contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5. 3) Count the cells that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 4) Inject one forward payload pointer movement. Verify that the SUT does not drop any cells. 5) Inject one forward payload pointer movement every 1 second. Verify that the SUT does not drop any cells. 6) Discontinue the payload pointer movement. 7) Inject five forward payload pointer movements every 1 second. Verify that the SUT does not drop any cells. 8) Discontinue the payload pointer movement. 9) Inject one backward payload pointer movement. Verify that the SUT does not drop any cells. 10) Inject one backward payload pointer movement every 1 second. Verify that the SUT does not drop any cells. 11) Discontinue the payload pointer movement. 12) Inject five backward payload pointer movements every 1 second. Verify that the SUT does not drop any cells. 13) Discontinue the payload pointer movement.
Reporting Format: The results of the cell loss due to pointer movement test SHOULD be reported in a form of a table. The rows SHOULD be labeled single pointer movement, one pointer movement per second, and five pointer movements per second. The columns SHOULD be labeled cell loss and number of cells lost. The elements of column 1 SHOULD be either True or False, indicating whether the particular condition was observed for each test. The elements of column 2 SHOULD be non-negative integers. The table MUST also indicate the traffic rate in IP PDUs per second as generated by the test device.3.1.1.3. IP Packet Loss due to Pointer Movement.
Objective: To determine if the SUT will drop IP packets due to pointer movements as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of IP packets at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 3) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 4) Inject one forward payload pointer movement. Verify that the SUT does not drop any packets. 5) Inject one forward payload pointer movement every 1 second. Verify that the SUT does not drop any packets. 6) Discontinue the payload pointer movement. 7) Inject five forward payload pointer movements every 1 second. Verify that the SUT does not drop any packets. 8) Discontinue the payload pointer movement.
9) Inject one backward payload pointer movement. Verify that the SUT does not drop any packets. 10) Inject one backward payload pointer movement every 1 second. Verify that the SUT does not drop any packets. 11) Discontinue the payload pointer movement. 12) Inject five backward payload pointer movements every 1 second. Verify that the SUT does not drop any packets. 13) Discontinue the payload pointer movement. Reporting Format: The results of the IP packet loss due to pointer movement test SHOULD be reported in a form of a table. The rows SHOULD be labeled single pointer movement, one pointer movement per second, and five pointer movements per second. The columns SHOULD be labeled packet loss and number of packets lost. The elements of column 1 SHOULD be either True or False, indicating whether the particular condition was observed for each test. The elements of column 2 SHOULD be non-negative integers. The table MUST also indicate the packet size in octets and traffic rate in packets per second as generated by the test device.3.1.2. Transport Overhead (TOH) Error Count
3.1.2.1. TOH Error Propagation.
Objective: To determine that the SUT does not propagate TOH errors as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of IP PDUs at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The cell payload SHOULD contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5. 3) Count the IP PDUs that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test, else lower the test device traffic rate until the counts are the same.
4) Inject one error in the first bit of the A1 and A2 Frameword. Verify that the SUT does not propagate the error. 5) Inject one error in the first bit of the A1 and A2 Frameword every 1 second. Verify that the SUT does not propagate the error. 6) Discontinue the Frameword error. 7) Inject one error in the first bit of the A1 and A2 Frameword for 4 consecutive IP PDUs in every 6 IP PDUs. Verify that the SUT indicates Loss of Frame. 8) Discontinue the Frameword error. Reporting Format: The results of the TOH error propagation test SHOULD be reported in a form of a table. The rows SHOULD be labeled single error, one error per second, and four consecutive errors every 6 IP PDUs. The columns SHOULD be labeled error propagated and loss of IP PDU. The elements of the table SHOULD be either True or False, indicating whether the particular condition was observed for each test. The table MUST also indicate the IP PDU size in octets and traffic rate in IP PDUs per second as generated by the test device.3.1.2.2. Cell Loss due to TOH Error
Objective: To determine if the SUT will drop cells due TOH Errors as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of cells at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The cell payload SHOULD contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5. 3) Count the cells that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same.
4) Inject one error in the first bit of the A1 and A2 Frameword. Verify that the SUT does not drop any cells. 5) Inject one error in the first bit of the A1 and A2 Frameword every 1 second. Verify that the SUT does not drop any cells. 6) Discontinue the Frameword error. 7) Inject one error in the first bit of the A1 and A2 Frameword for 4 consecutive IP PDUs in every 6 IP PDUs. Verify that the SUT does drop cells. 8) Discontinue the Frameword error. Reporting Format: The results of the Cell Loss due to TOH errors test SHOULD be reported in a form of a table. The rows SHOULD be labeled single error, one error per second, and four consecutive errors every 6 IP PDUs. The columns SHOULD be labeled cell loss and number of cells lost. The elements of column 1 SHOULD be either True or False, indicating whether the particular condition was observed for each test. The elements of column 2 SHOULD be non-negative integers. The table MUST also indicate the traffic rate in IP PDUs per second as generated by the test device.3.1.2.3. IP Packet Loss due to TOH Error.
Objective: To determine if the SUT will drop IP packets due to TOH errors as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of IP packets at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 3) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same.
4) Inject one error in the first bit of the A1 and A2 Frameword. Verify that the SUT does not drop any packets. 5) Inject one error in the first bit of the A1 and A2 Frameword every 1 second. Verify that the SUT does not drop any packets. 6) Discontinue the Frameword error. 7) Inject one error in the first bit of the A1 and A2 Frameword for 4 consecutive IP PDUs in every 6 IP PDUs. Verify that the SUT does drop packets. 8) Discontinue the Frameword error. Reporting Format: The results of the IP packet loss due to TOH errors test SHOULD be reported in a form of a table. The rows SHOULD be labeled single error, one error per second, and four consecutive errors every 6 IP PDUs. The columns SHOULD be labeled packet loss and number of packets lost. The elements of column 1 SHOULD be either True or False, indicating whether the particular condition was observed for each test. The elements of column 2 SHOULD be non-negative integers. The table MUST also indicate the packet size in octets and traffic rate in packets per second as generated by the test device.3.1.3. Path Overhead (POH) Error Count
3.1.3.1. POH Error Propagation.
Objective: To determine that the SUT does not propagate POH errors as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of IP PDUs at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The cell payload SHOULD contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5.
3) Count the IP PDUs that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test, else lower the test device traffic rate until the counts are the same. 4) Inject one error in the B3 (Path BIP8) byte. Verify that the SUT does not propagate the error. 5) Inject one error in the B3 byte every 1 second. Verify that the SUT does not propagate the error. 6) Discontinue the POH error. Reporting Format: The results of the POH error propagation test SHOULD be reported in a form of a table. The rows SHOULD be labeled single error and one error per second. The columns SHOULD be labeled error propagated and loss of IP PDU. The elements of the table SHOULD be either True or False, indicating whether the particular condition was observed for each test. The table MUST also indicate the IP PDU size in octets and traffic rate in IP PDUs per second as generated by the test device.3.1.3.2. Cell Loss due to POH Error.
Objective: To determine if the SUT will drop cells due POH Errors as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of cells at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The cell payload SHOULD contain valid IP PDUs. The IP PDUs MUST be encapsulated in AAL5. 3) Count the cells that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 4) Inject one error in the B3 (Path BIP8) byte. Verify that the SUT does not drop any cells.
5) Inject one error in the B3 byte every 1 second. Verify that the SUT does not drop any cells. 6) Discontinue the POH error. Reporting Format: The results of the Cell Loss due to POH errors test SHOULD be reported in a form of a table. The rows SHOULD be labeled single error and one error per second. The columns SHOULD be labeled cell loss and number of cells lost. The elements of column 1 SHOULD be either True or False, indicating whether the particular condition was observed for each test. The elements of column 2 SHOULD be non-negative integers. The table MUST also indicate the traffic rate in IP PDUs per second as generated by the test device.3.1.3.3. IP Packet Loss due to POH Error.
Objective: To determine if the SUT will drop IP packets due to POH errors as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the uni-directional configuration. 2) Send a specific number of IP packets at a specific rate through the SUT. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 3) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 4) Inject one error in the B3 (Path BIP8) byte. Verify that the SUT does not drop any packets. 5) Inject one error in the B3 byte every 1 second. Verify that the SUT does not drop any packets. 6) Discontinue the POH error.
Reporting Format: The results of the IP packet loss due to POH errors test SHOULD be reported in a form of a table. The rows SHOULD be labeled single error and one error per second. The columns SHOULD be labeled packet loss and number of packets lost. The elements of column 1 SHOULD be either True or False, indicating whether the particular condition was observed for each test. The elements of column 2 SHOULD be non-negative integers. The table MUST also indicate the packet size in octets and traffic rate in packets per second as generated by the test device.3.2. ATM Layer
3.2.1. Two-Point Cell Delay Variation (CDV)
3.2.1.1. Test Setup
The cell delay measurements assume that both the transmitter and receiver timestamp information is synchronized. Synchronization SHOULD be achieved by supplying a common clock signal (minimum of 100 Mhz or 10 ns resolution) to both the transmitter and receiver. The maximum timestamp values MUST be recorded to ensure synchronization in the case of counter rollover. The cell delay measurements SHOULD utilize the O.191 cell (ITUT-O.191) encapsulated in a valid IP packet. If the O.191 cell is not available, a test cell encapsulated in a valid IP packet MAY be used. The test cell MUST contain a transmit timestamp which can be correlated with a receive timestamp. A description of the test cell MUST be included in the test results. The description MUST include the timestamp length (in bits), counter rollover value, and the timestamp accuracy (in ns).3.2.1.2. Two-point CDV/Steady Load/One VCC
Objective: To determine the SUT variation in cell transfer delay with one VCC as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with one VCC. The VCC SHOULD contain one VPI/VCI. The VCC MUST be configured as either a CBR, VBR, or UBR connection. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]).
3) Send a specific number of IP packets containing timestamps at a specific constant rate through the SUT via the defined test VCC. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device. Reporting Format: The results of the Two-point CDV/Steady Load/One VCC test SHOULD be reported in a form of text, graph, and histogram. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, maximum and minimum CDV during the test in us, and peak-to-peak CDV in us. The graph results SHOULD display the cell delay values. The x- coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x- coordinate time SHOULD be configurable. The y-coordinate SHOULD be the cell delay in us. The integration time per point MUST be indicated. The histogram results SHOULD display the peak-to-peak cell delay. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.3. Two-point CDV/Steady Load/Twelve VCCs
Objective: To determine the SUT variation in cell transfer delay with twelve VCCs as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with twelve VCCs, using 1 VPI and 12 VCIs. The VCC's MUST be configured as either a CBR, VBR, or UBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps at a specific constant rate through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCCs. Reporting Format: The results of the Two-point CDV/Steady Load/Twelve VCCs test SHOULD be reported in a form of text, graph, and histograms. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI values, total number of cells transmitted and received on each VCC during the test in positive integers, maximum and minimum CDV on each VCC during the test in us, and peak-to-peak CDV on each VCC in us. The graph results SHOULD display the cell delay values. The x- coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x- coordinate time SHOULD be configurable. The y-coordinate SHOULD be the cell delay for each VCC in ms. There SHOULD be 12 curves on the graph, one curves indicated and labeled for each VCC. The integration time per point MUST be indicated.
The histograms SHOULD display the peak-to-peak cell delay. There will be one histogram for each VCC. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.4. Two-point CDV/Steady Load/Maximum VCCs
Objective: To determine the SUT variation in cell transfer delay with the maximum number VCCs supported on the SUT as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with the maximum number of VCCs supported on the SUT. For example, if the maximum number of VCCs supported on the SUT is 1024, define 256 VPIs with 4 VCIs per VPI. The VCC's MUST be configured as either a CBR, VBR, or UBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps at a specific constant rate through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCCs. Reporting Format: The results of the Two-point CDV/Steady Load/Maximum VCCs test SHOULD be reported in a form of text, graphs, and histograms.
The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI values, total number of cells transmitted and received on each VCC during the test in positive integers, maximum and minimum CDV on each VCC during the test in us, and peak-to-peak CDV on each VCC in us. The graph results SHOULD display the cell delay values. There will be (Max number of VCCs/10) graphs, with 10 VCCs indicated on each graph. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y- coordinate SHOULD be the cell delay for each VCC in us. There SHOULD be no more than 10 curves on each graph, one curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The histograms SHOULD display the peak-to-peak cell delay. There will be one histogram for each VCC. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.5. Two-point CDV/Bursty VBR Load/One VCC
Objective: To determine the SUT variation in cell transfer delay with one VCC as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with one VCC. The VCC SHOULD contain one VPI/VCI. The VCC MUST be configured as either a CBR or VBR connection. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps at a specific VBR through the SUT via the defined test VCC. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5.
4) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device. Reporting Format: The results of the Two-point CDV/Bursty VBR Load/One VCC test SHOULD be reported in a form of text, graph, and histogram. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, maximum and minimum CDV during the test in us, and peak-to-peak CDV in us. The graph results SHOULD display the cell delay values. The x- coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x- coordinate time SHOULD be configurable. The y-coordinate SHOULD be the cell delay in us. The integration time per point MUST be indicated. The histogram results SHOULD display the peak-to-peak cell delay. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.6. Two-point CDV/Bursty VBR Load/Twelve VCCs
Objective: To determine the SUT variation in cell transfer delay with twelve VCCs as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration.
2) Configure the SUT and test device with twelve VCCs, using 1 VPI and 12 VCIs. The VCC's MUST be configured as either a CBR or VBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps at a specific VBR through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCCs. Reporting Format: The results of the Two-point CDV/Bursty VBR Load/Twelve VCCs test SHOULD be reported in a form of text, graph, and histograms. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI values, total number of cells transmitted and received on each VCC during the test in positive integers, maximum and minimum CDV on each VCC during the test in us, and peak-to-peak CDV on each VCC in us. The graph results SHOULD display the cell delay values. The x- coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x- coordinate time SHOULD be configurable. The y-coordinate SHOULD be the cell delay for each VCC in ms. There SHOULD be 12 curves on the graph, one curves indicated and labeled for each VCC. The integration time per point MUST be indicated. The histograms SHOULD display the peak-to-peak cell delay. There will be one histogram for each VCC. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin.
The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.7. Two-point CDV/Bursty VBR Load/Maximum VCCs
Objective: To determine the SUT variation in cell transfer delay with the maximum number VCCs supported on the SUT as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with the maximum number of VCCs supported on the SUT. For example, if the maximum number of VCCs supported on the SUT is 1024, define 256 VPIs with 4 VCIs per VPI. The VCC's MUST be configured as either a CBR or VBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps at a specific VBR through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCCs. Reporting Format: The results of the Two-point CDV/Bursty VBR Load/Maximum VCCs test SHOULD be reported in a form of text, graphs, and histograms. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI values, total number of cells transmitted and received on
each VCC during the test in positive integers, maximum and minimum CDV on each VCC during the test in us, and peak-to-peak CDV on each VCC in us. The graph results SHOULD display the cell delay values. There will be (Max number of VCCs/10) graphs, with 10 VCCs indicated on each graph. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y- coordinate SHOULD be the cell delay for each VCC in us. There SHOULD be no more than 10 curves on each graph, one curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The histograms SHOULD display the peak-to-peak cell delay. There will be one histogram for each VCC. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.8. Two-point CDV/Mixed Load/Three VCC's
Objective: To determine the SUT variation in cell transfer delay with three VCC's as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with three VCC's. Each VCC MUST be defined as a different Bearer class: one CBR, one UBR and one VBR. Each VCC SHOULD contain one VPI/VCI. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps through the SUT via the defined test VCCs. Each generated VCC stream MUST match the corresponding VCC Bearer class. All of the VPI/VCI pairs will generate traffic at the same traffic rate.
Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCC's. Reporting Format: The results of the Two-point CDV/Mixed Load/Three VCC test SHOULD be reported in a form of text, graph, and histogram. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, maximum and minimum CDV during the test in us, and peak-to-peak CDV in us. The graph results SHOULD display the cell delay values. The x- coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x- coordinate time SHOULD be configurable. The y-coordinate SHOULD be the cell delay in us. The integration time per point MUST be indicated. The histogram results SHOULD display the peak-to-peak cell delay. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.9. Two-point CDV/Mixed Load/Twelve VCCs
Objective: To determine the SUT variation in cell transfer delay with twelve VCCs as defined in RFC 2761 "Terminology for ATM Benchmarking".
Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with twelve VCC's. Each VCC MUST be defined as one of the Bearer classes for a total of four CBR, four UBR and four VBR VCC's. Each VCC SHOULD contain one VPI/VCI. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps through the SUT via the defined test VCCs. Each generated VCC stream MUST match the corresponding VCC Bearer class. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCCs. Reporting Format: The results of the Two-point CDV/Mixed Load/Twelve VCCs test SHOULD be reported in a form of text, graph, and histograms. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI values, total number of cells transmitted and received on each VCC during the test in positive integers, maximum and minimum CDV on each VCC during the test in us, and peak-to-peak CDV on each VCC in us. The graph results SHOULD display the cell delay values. The x- coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x- coordinate time SHOULD be configurable. The y-coordinate SHOULD be the cell delay for each VCC in ms. There SHOULD be 12 curves on the graph, one curves indicated and labeled for each VCC. The integration time per point MUST be indicated.
The histograms SHOULD display the peak-to-peak cell delay. There will be one histogram for each VCC. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.1.10. Two-point CDV/Mixed Load/Maximum VCCs
Objective: To determine the SUT variation in cell transfer delay with the maximum number VCCs supported on the SUT as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with maximum number of VCCs supported on the SUT. For example, if the maximum number of VCCs supported on the SUT is 1024, define 256 VPIs with 4 VCIs per VPI. Each VCC MUST be defined as one of the Bearer classes for a total of (max VCC/3) CBR, (max VCC/3) UBR and (max VCC/3) VBR VCC's. If the maximum number of VCC's is not divisible by 3, the total for each bearer class MUST be within 3 VCC's of each other. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing timestamps through the SUT via the defined test VCCs. Each generated VCC stream MUST match the corresponding VCC Bearer class. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the packets timestamps at the transmitter and receiver ends of the test device for all VCCs.
Reporting Format: The results of the Two-point CDV/Mixed Load/Maximum VCCs test SHOULD be reported in a form of text, graphs, and histograms. The text results SHOULD display the numerical values of the CDV. The values given SHOULD include: time period of test in s, test VPI/VCI values, total number of cells transmitted and received on each VCC during the test in positive integers, maximum and minimum CDV on each VCC during the test in us, and peak-to-peak CDV on each VCC in us. The graph results SHOULD display the cell delay values. There will be (Max number of VCCs/10) graphs, with 10 VCCs indicated on each graph. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y- coordinate SHOULD be the cell delay for each VCC in us. There SHOULD be no more than 10 curves on each graph, one curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The histograms SHOULD display the peak-to-peak cell delay. There will be one histogram for each VCC. The x-coordinate SHOULD be the cell delay in us with at least 256 bins. The y-coordinate SHOULD be the number of cells observed in each bin. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST also be indicated.3.2.2. Cell Error Ratio (CER)
3.2.2.1. Test Setup
The cell error ratio measurements assume that both the transmitter and receiver payload information is synchronized. Synchronization MUST be achieved by supplying a known bit pattern to both the transmitter and receiver. If this bit pattern is longer than the packet size, the receiver MUST synchronize with the transmitter before tests can be run.
3.2.2.2. CER/Steady Load/One VCC
Objective: To determine the SUT ratio of errored cells on one VCC in a transmission in relation to the total cells sent as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with one VCC. The VCC SHOULD contain one VPI/VCI. The VCC MUST be configured as either a CBR, VBR, or UBR connection. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing one of the specified bit patterns at a constant rate through the SUT via the defined test VCC. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the number of bit errors at the receiver end of the test device. Reporting Format: The results of the CER/Steady Load/One VCC test SHOULD be reported in a form of text and graph. The text results SHOULD display the numerical values of the CER. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, and the CER for the entire test. The graph results SHOULD display the cell error ratio values. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y-coordinate SHOULD be the CER. The integration time per point MUST be indicated.
The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST be indicated. The generated bit pattern MUST also be indicated.3.2.2.3. CER/Steady Load/Twelve VCCs
Objective: To determine the SUT ratio of errored cells on twelve VCCs in a transmission in relation to the total cells sent as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with twelve VCCs, using 1 VPI and 12 VCIs. The VCC's MUST be configured as either a CBR, VBR, or UBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing one of the specified bit patterns at a constant rate through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the number of bit errors at the receiver end of the test device for all VCCs. Reporting Format: The results of the CER/Steady Load/Twelve VCCs test SHOULD be reported in a form of text and graph. The text results SHOULD display the numerical values of the CER. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, and the CER for the entire test.
The graph results SHOULD display the cell error ratio values. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y-coordinate SHOULD be the CER for each VCC. There should be 12 curves on the graph, on curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST be indicated. The generated bit pattern MUST also be indicated.3.2.2.4. CER/Steady Load/Maximum VCCs
Objective: To determine the SUT ratio of errored cells with the maximum number VCCs supported on the SUT in a transmission in relation to the total cells sent as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with the maximum number of VCCs supported on the SUT. For example, if the maximum number of VCCs supported on the SUT is 1024, define 256 VPIs with 4 VCIs per VPI. The VCC's MUST be configured as either a CBR, VBR, or UBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). 3) Send a specific number of IP packets containing one of the specified bit patterns at a constant rate through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the number of bit errors at the receiver end of the test device for all VCCs.
Reporting Format: The results of the CER/Steady Load/Maximum VCCs test SHOULD be reported in a form of text and graph. The text results SHOULD display the numerical values of the CER. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, and the CER for the entire test. The graph results SHOULD display the cell error ratio values. There will be (Max number of VCCs/10) graphs, with 10 VCCs indicated on each graph. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y-coordinate SHOULD be the CER for each VCC. There SHOULD be no more than 10 curves on each graph, one curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST be indicated. The generated bit pattern MUST also be indicated.3.2.2.5. CER/Bursty VBR Load/One VCC
Objective: To determine the SUT ratio of errored cells on one VCC in a transmission in relation to the total cells sent as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with one VCC. The VCC SHOULD contain one VPI/VCI. The VCC MUST be configured as either a CBR or VBR connection. The VPI/VCI MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). The PCR, SCR, and MBS must be configured using one of the specified traffic descriptors.
3) Send a specific number of IP packets containing one of the specified bit patterns at a specific VBR rate through the SUT via the defined test VCC. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the number of bit errors at the receiver end of the test device. Reporting Format: The results of the CER/Bursty VBR Load/One VCC test SHOULD be reported in a form of text and graph. The text results SHOULD display the numerical values of the CER. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, and the CER for the entire test. The graph results SHOULD display the cell error ratio values. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y-coordinate SHOULD be the CER. The integration time per point MUST be indicated. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST be indicated. The generated bit pattern MUST also be indicated.3.2.2.6. CER/Bursty VBR Load/Twelve VCCs
Objective: To determine the SUT ratio of errored cells on twelve VCCs in a transmission in relation to the total cells sent as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration.
2) Configure the SUT and test device with twelve VCCs, using 1 VPI and 12 VCIs. The VCC's MUST be configured as either a CBR or VBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). The PCR, SCR, and MBS must be configured using one of the specified traffic descriptors. 3) Send a specific number of IP packets containing one of the specified bit patterns at a specific VBR rate through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The PCR, SCR, and MBS must be indicated. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the number of bit errors at the receiver end of the test device for all VCCs. Reporting Format: The results of the CER/Bursty VBR Load/Twelve VCCs test SHOULD be reported in a form of text and graph. The text results SHOULD display the numerical values of the CER. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, and the CER for the entire test. The graph results SHOULD display the cell error ratio values. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y-coordinate SHOULD be the CER for each VCC. There should be 12 curves on the graph, on curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST be indicated. The generated bit pattern MUST also be indicated.
3.2.2.7. CER/Bursty VBR Load/Maximum VCCs
Objective: To determine the SUT ratio of errored cells with the maximum number VCCs supported on the SUT in a transmission in relation to the total cells sent as defined in RFC 2761 "Terminology for ATM Benchmarking". Procedure: 1) Set up the SUT and test device using the bi-directional configuration. 2) Configure the SUT and test device with the maximum number of VCCs supported on the SUT. For example, if the maximum number of VCCs supported on the SUT is 1024, define 256 VPIs with 4 VCIs per VPI. The VCC's MUST be configured as either a CBR or VBR connection. The VPI/VCIs MUST not be one of the reserved ATM signaling channels (e.g., [0,5], [0,16]). The PCR, SCR, and MBS must be configured using one of the specified traffic descriptors. 3) Send a specific number of IP packets containing one of the specified bit patterns at a specific VBR rate through the SUT via the defined test VCCs. All of the VPI/VCI pairs will generate traffic at the same traffic rate. Since this test is not a throughput test, the rate should not be greater than 90% of line rate. The IP PDUs MUST be encapsulated in AAL5. 4) Count the IP packets that are transmitted by the SUT on all VCCs to verify connectivity and load. If the count on the test device is the same on the SUT, continue the test; else lower the test device traffic rate until the counts are the same. 5) Record the number of bit errors at the receiver end of the test device for all VCCs. Reporting Format: The results of the CER/Bursty VBR Load/Maximum VCCs test SHOULD be reported in a form of text and graph. The text results SHOULD display the numerical values of the CER. The values given SHOULD include: time period of test in s, test VPI/VCI value, total number of cells transmitted and received on the given VPI/VCI during the test in positive integers, and the CER for the entire test.
The graph results SHOULD display the cell error ratio values. There will be (Max number of VCCs/10) graphs, with 10 VCCs indicated on each graph. The x-coordinate SHOULD be the test run time in either seconds, minutes or days depending on the total length of the test. The x-coordinate time SHOULD be configurable. The y-coordinate SHOULD be the CER for each VCC. There SHOULD be no more than 10 curves on each graph, one curve indicated and labeled for each VCC. The integration time per point MUST be indicated. The results MUST also indicate the packet size in octets, traffic rate in packets per second, and bearer class as generated by the test device. The VCC and VPI/VCI values MUST be indicated. The PCR, SCR, and MBS MUST be indicated. The bearer class of the created VCC MUST be indicated. The generated bit pattern MUST also be indicated.