FD.io CSIT-2106 Release Report is published


Maciek Konstantynowicz (mkonstan)
 

Hi All,

FD.io CSIT-2106 report is now available on FD.io docs site:

https://docs.fd.io/csit/rls2106/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2106 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to csit-dev@lists.fd.io.

Cheers,
Maciek (CSIT PTL)


CSIT-2106 Release Summary
-------------------------

BENCHMARK TESTS

- AF_XDP: Added af_xdp driver support for all test cases. Test results
will be added in sub-sequent CSIT-2106 report.

- GTPU tunnel: Added GTPU HW Offload IPv4 routing tests.

- Intel Xeon Ice Lake: Added initial test data for these platforms.
Current CSIT-2106 report data for Intel Xeon Ice Lake comes from an
external source (Intel labs running CSIT code on
"8360Y D Stepping" and "6338N" processors).

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
environment. Test results will be added in sub-sequent CSIT-2106
report.

- MLRsearch improvements: Added support for multiple packet throughput
rates in a single search, each rate is associated with a distinct
Packet Loss Ratio (PLR) criterion. Implemented number of
optimizations improving rate discovery efficiency.

TEST FRAMEWORK

- Telemetry retouch: Refactored telemetry retrieval from DUTs and SUTs.
Included VPP perfmon plugin telemetry with all perfmon bundles
available in VPP release.

- Upgrade to Ubuntu 20.04 LTS: Re-installed base operating system to
Ubuntu 20.04.2 LTS. Upgrade included also baseline Docker containers
used for spawning topology.

Pointers to CSIT-2106 Report sections
-------------------------------------

1. FD.io CSIT test methodology [1]
2. VPP release notes [2]
3. VPP 64B/IMIX throughput graphs [3]
4. VPP throughput speedup multi-core [4]
5. VPP latency under load [5]
6. VPP comparisons v21.06 vs. v21.01 [6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 20.08 apps release notes [8]
9. DPDK 64B throughput graphs [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.02 vs. 20.11 [11]

Functional device tests (VPP_Device) are also included in the report.

[1] https://docs.fd.io/csit/rls2106/report/introduction/methodology.html
[2] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/csit_release_notes.html
[3] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_latency/index.html
[6] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] https://docs.fd.io/csit/rls2106/report/detailed_test_results/vpp_performance_results/index.html
[8] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/csit_release_notes.html
[9] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[10] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_latency/index.html
[11] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html


Maciek Konstantynowicz (mkonstan)
 

Caveat note :)

Number of the latency graphs in report show excessive numbers O(sec)
across frame sizes w/o and with load for both VPP and DPDK.

Few of us in CSIT are working on the resolution, stay tuned.
For now please ignore those numbers.

Cheers,
Maciek

On 15 Jul 2021, at 13:03, Maciek Konstantynowicz (mkonstan) <mkonstan@cisco.com> wrote:

Hi All,

FD.io CSIT-2106 report is now available on FD.io docs site:

https://docs.fd.io/csit/rls2106/report/

Another successful release! Many thanks to all contributors in CSIT and
VPP communities for making it happen.

See below for CSIT-2106 release summary and pointers to specific
sections in the report.

Welcome all comments, best by email to csit-dev@lists.fd.io.

Cheers,
Maciek (CSIT PTL)


CSIT-2106 Release Summary
-------------------------

BENCHMARK TESTS

- AF_XDP: Added af_xdp driver support for all test cases. Test results
will be added in sub-sequent CSIT-2106 report.

- GTPU tunnel: Added GTPU HW Offload IPv4 routing tests.

- Intel Xeon Ice Lake: Added initial test data for these platforms.
Current CSIT-2106 report data for Intel Xeon Ice Lake comes from an
external source (Intel labs running CSIT code on
"8360Y D Stepping" and "6338N" processors).

- CSIT in AWS environment: Added CSIT support for AWS c5n instances
environment. Test results will be added in sub-sequent CSIT-2106
report.

- MLRsearch improvements: Added support for multiple packet throughput
rates in a single search, each rate is associated with a distinct
Packet Loss Ratio (PLR) criterion. Implemented number of
optimizations improving rate discovery efficiency.

TEST FRAMEWORK

- Telemetry retouch: Refactored telemetry retrieval from DUTs and SUTs.
Included VPP perfmon plugin telemetry with all perfmon bundles
available in VPP release.

- Upgrade to Ubuntu 20.04 LTS: Re-installed base operating system to
Ubuntu 20.04.2 LTS. Upgrade included also baseline Docker containers
used for spawning topology.

Pointers to CSIT-2106 Report sections
-------------------------------------

1. FD.io CSIT test methodology [1]
2. VPP release notes [2]
3. VPP 64B/IMIX throughput graphs [3]
4. VPP throughput speedup multi-core [4]
5. VPP latency under load [5]
6. VPP comparisons v21.06 vs. v21.01 [6]
7. VPP performance all pkt sizes & NICs [7]
8. DPDK 20.08 apps release notes [8]
9. DPDK 64B throughput graphs [9]
10. DPDK latency under load [10]
11. DPDK comparisons 21.02 vs. 20.11 [11]

Functional device tests (VPP_Device) are also included in the report.

[1] https://docs.fd.io/csit/rls2106/report/introduction/methodology.html
[2] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/csit_release_notes.html
[3] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_throughput_graphs/index.html
[4] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/throughput_speedup_multi_core/index.html
[5] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/packet_latency/index.html
[6] https://docs.fd.io/csit/rls2106/report/vpp_performance_tests/comparisons/current_vs_previous_release.html
[7] https://docs.fd.io/csit/rls2106/report/detailed_test_results/vpp_performance_results/index.html
[8] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/csit_release_notes.html
[9] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_throughput_graphs/index.html
[10] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/packet_latency/index.html
[11] https://docs.fd.io/csit/rls2106/report/dpdk_performance_tests/comparisons/current_vs_previous_release.html