NSF SFC tests in CSIT


Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
 

Hi

 

We started the NSH SFC functional tests in CSIT for the Report rls1901 and all of them failed, see:

 

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/2/

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-centos7-virl/2/

 

Can you look at it and let me know if:

1.       You will fix them so we can run them and include the results to the report, or

2.       We can completely remove NSH SFC from the Report

 

Thank you

Tibor

 


Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>
 

Hello,

 

I did correction to download correct vpp in nsh_sfc csit job: https://gerrit.fd.io/r/#/c/17598/

 

And triggered new run: https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/3/

 

6 tests are still failing – issue is in setup DUT nodes for SFF func testing – some cli commands are failing:

 

exec_command on ('10.30.53.91', 22) with timeout 600: cd /tmp/openvpp-testing/tests/nsh_sfc/sfc_scripts/ && sudo ./set_sfc_sff.sh GigabitEthernet0/5/0 GigabitEthernet0/4/0 fa:16:3e:ad:7c:6b fa:16:3e:f4:6d:fe

exec_command on ('10.30.53.91', 22) took 27.2735719681 seconds

return RC 0

return STDOUT set interface ip table: no such table 7

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel0 1 1'

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel1 1 1'

create nsh map: parse error: 'encap-vxlan-gpe-intf'

set interface ip table: no such table 7

set ip arp: fib ID 7 doesn't exist

 

Regards,

Jan

 

From: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Sent: Thursday, February 14, 2019 8:58 AM
To: nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>; Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>
Subject: NSF SFC tests in CSIT

 

Hi

 

We started the NSH SFC functional tests in CSIT for the Report rls1901 and all of them failed, see:

 

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/2/

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-centos7-virl/2/

 

Can you look at it and let me know if:

1.      You will fix them so we can run them and include the results to the report, or

2.      We can completely remove NSH SFC from the Report

 

Thank you

Tibor

 


Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>
 

Hello,

 

Those tests should be either rewritten using CSIT framework (PAPI, not some bash scripts with static configuration) and maintained or removed.

The way how tests are written (as a series of bash scripts) is not supported way in CSIT. They are hard to debug and maintain.

 

Peter Mikus
Engineer – Software

Cisco Systems Limited

 

From: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>
Sent: Thursday, February 14, 2019 10:32 AM
To: nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>; Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Subject: RE: NSF SFC tests in CSIT

 

Hello,

 

I did correction to download correct vpp in nsh_sfc csit job: https://gerrit.fd.io/r/#/c/17598/

 

And triggered new run: https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/3/

 

6 tests are still failing – issue is in setup DUT nodes for SFF func testing – some cli commands are failing:

 

exec_command on ('10.30.53.91', 22) with timeout 600: cd /tmp/openvpp-testing/tests/nsh_sfc/sfc_scripts/ && sudo ./set_sfc_sff.sh GigabitEthernet0/5/0 GigabitEthernet0/4/0 fa:16:3e:ad:7c:6b fa:16:3e:f4:6d:fe

exec_command on ('10.30.53.91', 22) took 27.2735719681 seconds

return RC 0

return STDOUT set interface ip table: no such table 7

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel0 1 1'

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel1 1 1'

create nsh map: parse error: 'encap-vxlan-gpe-intf'

set interface ip table: no such table 7

set ip arp: fib ID 7 doesn't exist

 

Regards,

Jan

 

From: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Sent: Thursday, February 14, 2019 8:58 AM
To: nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>; Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>
Subject: NSF SFC tests in CSIT

 

Hi

 

We started the NSH SFC functional tests in CSIT for the Report rls1901 and all of them failed, see:

 

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/2/

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-centos7-virl/2/

 

Can you look at it and let me know if:

1.       You will fix them so we can run them and include the results to the report, or

2.       We can completely remove NSH SFC from the Report

 

Thank you

Tibor

 


Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
 

I agree with Peter. Any comments from NSH SFC dev team?

 

From: Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>
Sent: Thursday, February 14, 2019 11:21 AM
To: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>; nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Subject: RE: NSF SFC tests in CSIT

 

Hello,

 

Those tests should be either rewritten using CSIT framework (PAPI, not some bash scripts with static configuration) and maintained or removed.

The way how tests are written (as a series of bash scripts) is not supported way in CSIT. They are hard to debug and maintain.

 

Peter Mikus
Engineer – Software

Cisco Systems Limited

 

From: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>
Sent: Thursday, February 14, 2019 10:32 AM
To: nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>; Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Subject: RE: NSF SFC tests in CSIT

 

Hello,

 

I did correction to download correct vpp in nsh_sfc csit job: https://gerrit.fd.io/r/#/c/17598/

 

And triggered new run: https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/3/

 

6 tests are still failing – issue is in setup DUT nodes for SFF func testing – some cli commands are failing:

 

exec_command on ('10.30.53.91', 22) with timeout 600: cd /tmp/openvpp-testing/tests/nsh_sfc/sfc_scripts/ && sudo ./set_sfc_sff.sh GigabitEthernet0/5/0 GigabitEthernet0/4/0 fa:16:3e:ad:7c:6b fa:16:3e:f4:6d:fe

exec_command on ('10.30.53.91', 22) took 27.2735719681 seconds

return RC 0

return STDOUT set interface ip table: no such table 7

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel0 1 1'

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel1 1 1'

create nsh map: parse error: 'encap-vxlan-gpe-intf'

set interface ip table: no such table 7

set ip arp: fib ID 7 doesn't exist

 

Regards,

Jan

 

From: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Sent: Thursday, February 14, 2019 8:58 AM
To: nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>; Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>
Subject: NSF SFC tests in CSIT

 

Hi

 

We started the NSH SFC functional tests in CSIT for the Report rls1901 and all of them failed, see:

 

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/2/

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-centos7-virl/2/

 

Can you look at it and let me know if:

1.       You will fix them so we can run them and include the results to the report, or

2.       We can completely remove NSH SFC from the Report

 

Thank you

Tibor

 


Ni, Hongjun
 

Hi Jan,

 

Thank you for your effort on NSH_SFC tests.

 

Since all features in NSH_SFC project has been moved to VPP main repo, and NSH_SFC is not maintained today,

We will develop new CSIT test cases based on new CSIT framework later and VPP NSH plugin.

Please remove NSH_SFC tests from CSIT 19.01 release.

 

Thanks,

Hongjun

 

From: nsh_sfc-dev@... [mailto:nsh_sfc-dev@...] On Behalf Of via Lists.Fd.Io
Sent: Thursday, February 14, 2019 5:32 PM
To: nsh_sfc-dev@...
Cc: nsh_sfc-dev@...
Subject: Re: [nsh_sfc-dev] NSF SFC tests in CSIT

 

Hello,

 

I did correction to download correct vpp in nsh_sfc csit job: https://gerrit.fd.io/r/#/c/17598/

 

And triggered new run: https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/3/

 

6 tests are still failing – issue is in setup DUT nodes for SFF func testing – some cli commands are failing:

 

exec_command on ('10.30.53.91', 22) with timeout 600: cd /tmp/openvpp-testing/tests/nsh_sfc/sfc_scripts/ && sudo ./set_sfc_sff.sh GigabitEthernet0/5/0 GigabitEthernet0/4/0 fa:16:3e:ad:7c:6b fa:16:3e:f4:6d:fe

exec_command on ('10.30.53.91', 22) took 27.2735719681 seconds

return RC 0

return STDOUT set interface ip table: no such table 7

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel0 1 1'

create vxlan-gpe tunnel: nonexistent encap fib id 7

set interface l2 bridge: unknown interface `vxlan_gpe_tunnel1 1 1'

create nsh map: parse error: 'encap-vxlan-gpe-intf'

set interface ip table: no such table 7

set ip arp: fib ID 7 doesn't exist

 

Regards,

Jan

 

From: Tibor Frank -X (tifrank - PANTHEON TECHNOLOGIES at Cisco) <tifrank@...>
Sent: Thursday, February 14, 2019 8:58 AM
To: nsh_sfc-dev@...
Cc: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) <jgelety@...>; Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco) <pmikus@...>
Subject: NSF SFC tests in CSIT

 

Hi

 

We started the NSH SFC functional tests in CSIT for the Report rls1901 and all of them failed, see:

 

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-ubuntu1604-virl/2/

https://jenkins.fd.io/job/csit-nsh_sfc-verify-func-1901-centos7-virl/2/

 

Can you look at it and let me know if:

1.       You will fix them so we can run them and include the results to the report, or

2.       We can completely remove NSH SFC from the Report

 

Thank you

Tibor