|
Re: FD.io Production Jenkins Restart Required
Maciek Konstantynowicz (mkonstan)
Hi Vanessa, Thanks for the note. CSIT project keeps experiencing issues
toggle quoted messageShow quoted text
due to Jenkins outages. Do you have ETA for the fix that will stop these outages? -Maciek
On 5 Nov 2019, at 23:18, Vanessa Valderrama <vvalderrama@linuxfoundation.org> wrote:
|
|
|
|
Re: FD.io Production Jenkins Restart Required
Vanessa Valderrama
Jenkins has been restarted, job views restored, jobs are running.
toggle quoted messageShow quoted text
We will continue to investigate the Gateway Timeout and JNLP errors we've been seeing the last couple of days. If you experience any issues, please open a ticket at support.linuxfoundation.org Thank you, Vanessa
On 11/5/19 4:39 PM, Vanessa Valderrama wrote:
We continue having issues with Gateway Timeouts on the CI merge job which has
|
|
|
|
FD.io Production Jenkins Restart Required
Vanessa Valderrama
We continue having issues with Gateway Timeouts on the CI merge job which has
corrupted the Jenkins job views. Jenkins will need to be restarted to resolve this issue. Thank you, Vanessa
|
|
|
|
Re: Jenkins Issue
Vanessa Valderrama
Jenkins is out of shutdown mode. Jobs are starting. If you experience
toggle quoted messageShow quoted text
any issues please open a ticket at support.linuxfoundation.org. Thank you, Vanessa
On 11/4/19 4:33 PM, Vanessa Valderrama wrote:
Jenkins has been restarted.
|
|
|
|
Re: Jenkins Issue
Vanessa Valderrama
Jenkins has been restarted.
toggle quoted messageShow quoted text
We're pushing the jobs now.
On 11/4/19 2:22 PM, Vanessa Valderrama wrote:
We are having issues with Gateway Timeouts on the CI merge job which has
|
|
|
|
Jenkins Issue
Vanessa Valderrama
We are having issues with Gateway Timeouts on the CI merge job which has
corrupted the Jenkins job views. We are aware aware of the issue and will have it resolved shortly. Jenkins will be placed in shutdown mode while we generate the jobs. Thank you, Vanessa
|
|
|
|
Re: FD.io Maintenance: 2019-10-23 1700 UTC to 2100 UTC
Vanessa Valderrama
The migration went much quicker towards the end. Maintenance is complete. All systems are available. Please open a ticket at support.linuxfoundation.org if you experience any issues. Thank you,Anton & Vanessa
On 10/23/2019 03:56 PM, Vanessa
Valderrama wrote:
|
|
|
|
Re: FD.io Maintenance: 2019-10-23 1700 UTC to 2100 UTC
Vanessa Valderrama
We need to extend the maintenance window until 2300 to allow the
volume migration to complete.
On 10/23/2019 11:10 AM, Vanessa
Valderrama wrote:
|
|
|
|
Re: FD.io Maintenance: 2019-10-23 1700 UTC to 2100 UTC
Vanessa Valderrama
Jenkins has been placed in shtudown mode in preparation for
maintenance.
On 10/22/2019 09:35 AM, Vanessa
Valderrama wrote:
|
|
|
|
Re: FD.io Maintenance: 2019-10-23 1700 UTC to 2100 UTC
Vanessa Valderrama
Maintenance reminder
On Oct 17, 2019, at 1:04 PM, Vanessa Valderrama <vvalderrama@...> wrote:
|
|
|
|
Catch the OVP Webinar on Tuesday, Oct 22, 10:00 AM PT
Brandon Wick <bwick@...>
LFN Community: For those not yet familiar, the OPNFV Verification Program (OVP) is an open source, community-led compliance and verification program to demonstrate the readiness and availability of commercial NFV products and services, including NFVI and VNFs, using OPNFV and ONAP components. LFN has contracted with SDxCentral in order to raise awareness of the program via a webinar this coming Tuesday at 10:00 AM PT How the OPNFV Verification Program (OVP) Can Boost VNF Interoperability. Community leaders Lincoln Lavoie, Rabi Abdel, and Amar Kapadia will provide an overview of the program, explain the benefits, and invite participation from the industry. If you'd like to learn more about the program, please register and join us on Tuesday. If you can't attend live, please register to be informed when the on-demand replay is available. Also, if you work with any industry partners that you feel might be interested in the webinar, please forward them the link: https://www.sdxcentral.com/resources/sponsored/webinars/linux-foundation-opnfv-verification-program-boosts-vnf-interoperability/. Thank you! Best, Brandon Wick
|
|
|
|
FD.io Nexus Maintenance: 2019-10-23 1700 UTC to 2100 UTC
Vanessa Valderrama
What: LF will be performing standard system maintenance and a Jenkins migration
When: 2019-10-23 1700 UTC to 2100 UTC Impact: Maintenance will require a reboot of each FD.io system. Jenkins will be placed in shutdown mode at 1600 UTC. Please let us know if specific jobs cannot be aborted. The following systems will be unavailable during the maintenance window:
|
|
|
|
Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
> only discovering this now.
Not sure if it is relevant, but scapy linking has been introduced to TRex [20], CSIT [21] and VPP [22] in 2016 (although for CSIT it was the first real commit in the open-source repo, so closed-source linking probably happened sooner), in what seems to me like three independent decisions.
Vratko.
[21] https://gerrit.fd.io/r/c/csit/+/107 [22] https://gerrit.fd.io/r/c/vpp/+/3237
From: Joel Halpern <joel.halpern@...>
Sent: Wednesday, October 16, 2019 4:30 PM To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>; Ray Kinsella <ray.kinsella@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Subject: RE: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
I have a real problem with important parts of fd.io (and CSIT is an important part both for the project and for consumers of the project) being tied to GPL.
I am really unhappy that we are only discovering this now.
Yours, Joel
From: tsc@... <tsc@...>
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
> [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. > I am not 100% certain it is still a FD.io project?
TRex is still FD.io project as far as I know. Git+Gerrit server at gerrit.fd.io is just a tool, projects are not required to use it.
The github repo is the official primary one for TRex, while CSIT and VPP still use gerrit.fd.io as the primary, and github just as a mirror.
> counts as distributing the software.
TRex offers releases for download, for example [19]. They are basically archives of git content at that time, including the problematic Python bits.
> Can we be clear that anything in CSIT that links/loads > scapy directly is GPL’ed (or a compatible license).
Looking at the Charter [18], we can do that according to 7.c, (giving us the exception to otherwise prohibiting 7.b.i and 7.b.iii). The downside is, around 90% or more of CSIT code would end up as GPL (almost everything is linked together).
Reducing the percentage by offloading the scapy processing into a separate process is of course possible, but it would also require a considerable effort to finish.
Anyway, switching parts of code to GPL quickly and then reducing the amount of code affected slowly looks like a good plan to me. 7.c just requires a request, justification, and two-thirds TSC vote.
Vratko.
[19] https://github.com/cisco-system-traffic-generator/trex-core/releases/tag/v2.61
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> Why is TRex’s licensing being questioned?
It just happened to be the first FD.io project examined (by Miroslav) for dependencies. VPP and CSIT were examined (by Vratko) few days later. [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. I am not 100% certain it is still a FD.io project?
> If we don’t redistribute, can’t we just take it’s licensing at face value?
We (CSIT) could do that, except that CSIT also uses scapy directly. [rk] Ok, can we focus and resolve on CSIT’s usage first [rk] Can we be clear that anything in CSIT that links/loads scapy directly is GPL’ed (or a compatible license). [rk] Anything else (non-GPL’ed) that talk’s to scapy via interfaces that preserve isolation?
We (FD.io) cannot do that, as TRex is a FD.io project which distributes TRex code, including the problematic Python bits. [rk] Ok – we need Ed to check this. [rk] In that case, should it not be someone from TRex covering it?
I believe even having the git repository publicly cloneable counts as distributing the software. [rk] Perhaps, but there has been no contributions to the FD.io VPP TRex git since 2017 [rk] We need to ask, what the best approach is here.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|
|
Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Maciek Konstantynowicz <mackonstan@...>
Correcting the email subject to make it clear it’s related to Scapy licensing agenda topic for today’s FD.io TSC meeting.
toggle quoted messageShow quoted text
Cheers,
-Maciek
|
|
|
|
TRex distribution and licensing of external libraries
Maciek Konstantynowicz <mackonstan@...>
Hi, Here is the background to Scapy licensing agenda topic for today’s FD.io TSC meeting.
Cheers,
-Maciek
|
|
|
|
Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Joel Halpern
I have a real problem with important parts of fd.io (and CSIT is an important part both for the project and for consumers of the project) being tied to GPL.
I am really unhappy that we are only discovering this now.
Yours, Joel
From: tsc@... <tsc@...> On Behalf Of
Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Sent: Wednesday, October 16, 2019 10:26 AM To: Ray Kinsella <ray.kinsella@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Cc: tsc@... Subject: Re: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
> [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. > I am not 100% certain it is still a FD.io project?
TRex is still FD.io project as far as I know. Git+Gerrit server at gerrit.fd.io is just a tool, projects are not required to use it.
The github repo is the official primary one for TRex, while CSIT and VPP still use gerrit.fd.io as the primary, and github just as a mirror.
> counts as distributing the software.
TRex offers releases for download, for example [19]. They are basically archives of git content at that time, including the problematic Python bits.
> Can we be clear that anything in CSIT that links/loads > scapy directly is GPL’ed (or a compatible license).
Looking at the Charter [18], we can do that according to 7.c, (giving us the exception to otherwise prohibiting 7.b.i and 7.b.iii). The downside is, around 90% or more of CSIT code would end up as GPL (almost everything is linked together).
Reducing the percentage by offloading the scapy processing into a separate process is of course possible, but it would also require a considerable effort to finish.
Anyway, switching parts of code to GPL quickly and then reducing the amount of code affected slowly looks like a good plan to me. 7.c just requires a request, justification, and two-thirds TSC vote.
Vratko.
[19] https://github.com/cisco-system-traffic-generator/trex-core/releases/tag/v2.61
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> Why is TRex’s licensing being questioned?
It just happened to be the first FD.io project examined (by Miroslav) for dependencies. VPP and CSIT were examined (by Vratko) few days later. [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. I am not 100% certain it is still a FD.io project?
> If we don’t redistribute, can’t we just take it’s licensing at face value?
We (CSIT) could do that, except that CSIT also uses scapy directly. [rk] Ok, can we focus and resolve on CSIT’s usage first [rk] Can we be clear that anything in CSIT that links/loads scapy directly is GPL’ed (or a compatible license). [rk] Anything else (non-GPL’ed) that talk’s to scapy via interfaces that preserve isolation?
We (FD.io) cannot do that, as TRex is a FD.io project which distributes TRex code, including the problematic Python bits. [rk] Ok – we need Ed to check this. [rk] In that case, should it not be someone from TRex covering it?
I believe even having the git repository publicly cloneable counts as distributing the software. [rk] Perhaps, but there has been no contributions to the FD.io VPP TRex git since 2017 [rk] We need to ask, what the best approach is here.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|
|
Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
> [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. > I am not 100% certain it is still a FD.io project?
TRex is still FD.io project as far as I know. Git+Gerrit server at gerrit.fd.io is just a tool, projects are not required to use it.
The github repo is the official primary one for TRex, while CSIT and VPP still use gerrit.fd.io as the primary, and github just as a mirror.
> counts as distributing the software.
TRex offers releases for download, for example [19]. They are basically archives of git content at that time, including the problematic Python bits.
> Can we be clear that anything in CSIT that links/loads > scapy directly is GPL’ed (or a compatible license).
Looking at the Charter [18], we can do that according to 7.c, (giving us the exception to otherwise prohibiting 7.b.i and 7.b.iii). The downside is, around 90% or more of CSIT code would end up as GPL (almost everything is linked together).
Reducing the percentage by offloading the scapy processing into a separate process is of course possible, but it would also require a considerable effort to finish.
Anyway, switching parts of code to GPL quickly and then reducing the amount of code affected slowly looks like a good plan to me. 7.c just requires a request, justification, and two-thirds TSC vote.
Vratko.
[19] https://github.com/cisco-system-traffic-generator/trex-core/releases/tag/v2.61
From: tsc@... <tsc@...> On Behalf Of
Ray Kinsella
Sent: Wednesday, October 16, 2019 1:32 PM To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Subject: Re: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> Why is TRex’s licensing being questioned?
It just happened to be the first FD.io project examined (by Miroslav) for dependencies. VPP and CSIT were examined (by Vratko) few days later. [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. I am not 100% certain it is still a FD.io project?
> If we don’t redistribute, can’t we just take it’s licensing at face value?
We (CSIT) could do that, except that CSIT also uses scapy directly. [rk] Ok, can we focus and resolve on CSIT’s usage first [rk] Can we be clear that anything in CSIT that links/loads scapy directly is GPL’ed (or a compatible license). [rk] Anything else (non-GPL’ed) that talk’s to scapy via interfaces that preserve isolation?
We (FD.io) cannot do that, as TRex is a FD.io project which distributes TRex code, including the problematic Python bits. [rk] Ok – we need Ed to check this. [rk] In that case, should it not be someone from TRex covering it?
I believe even having the git repository publicly cloneable counts as distributing the software. [rk] Perhaps, but there has been no contributions to the FD.io VPP TRex git since 2017 [rk] We need to ask, what the best approach is here.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|
|
Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Ray Kinsella
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
Sent: Wednesday 16 October 2019 12:10 To: Kinsella, Ray <ray.kinsella@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Subject: RE: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
> Why is TRex’s licensing being questioned?
It just happened to be the first FD.io project examined (by Miroslav) for dependencies. VPP and CSIT were examined (by Vratko) few days later. [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. I am not 100% certain it is still a FD.io project?
> If we don’t redistribute, can’t we just take it’s licensing at face value?
We (CSIT) could do that, except that CSIT also uses scapy directly. [rk] Ok, can we focus and resolve on CSIT’s usage first [rk] Can we be clear that anything in CSIT that links/loads scapy directly is GPL’ed (or a compatible license). [rk] Anything else (non-GPL’ed) that talk’s to scapy via interfaces that preserve isolation?
We (FD.io) cannot do that, as TRex is a FD.io project which distributes TRex code, including the problematic Python bits. [rk] Ok – we need Ed to check this. [rk] In that case, should it not be someone from TRex covering it?
I believe even having the git repository publicly cloneable counts as distributing the software. [rk] Perhaps, but there has been no contributions to the FD.io VPP TRex git since 2017 [rk] We need to ask, what the best approach is here.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From:
tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From:
tsc@... <tsc@...> On Behalf Of
Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From:
tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|
|
Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
> Why is TRex’s licensing being questioned?
It just happened to be the first FD.io project examined (by Miroslav) for dependencies. VPP and CSIT were examined (by Vratko) few days later.
> If we don’t redistribute, can’t we just take it’s licensing at face value?
We (CSIT) could do that, except that CSIT also uses scapy directly.
We (FD.io) cannot do that, as TRex is a FD.io project which distributes TRex code, including the problematic Python bits.
I believe even having the git repository publicly cloneable counts as distributing the software.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Sent: Tuesday, October 15, 2019 1:05 PM To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Subject: RE: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From:
tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|
|
Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
Ray Kinsella
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Sent: Monday 14 October 2019 14:20 To: Kinsella, Ray <ray.kinsella@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Cc: tsc@... Subject: Re: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From:
tsc@... <tsc@...> On Behalf Of
Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From:
tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|
