Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
|
Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
> only discovering this now.
Not sure if it is relevant, but scapy linking has been introduced to TRex [20], CSIT [21] and VPP [22] in 2016 (although for CSIT it was the first real commit in the open-source repo, so closed-source linking probably happened sooner), in what seems to me like three independent decisions.
Vratko.
[21] https://gerrit.fd.io/r/c/csit/+/107 [22] https://gerrit.fd.io/r/c/vpp/+/3237
From: Joel Halpern <joel.halpern@...>
Sent: Wednesday, October 16, 2019 4:30 PM To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>; Ray Kinsella <ray.kinsella@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...> Subject: RE: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]
I have a real problem with important parts of fd.io (and CSIT is an important part both for the project and for consumers of the project) being tied to GPL.
I am really unhappy that we are only discovering this now.
Yours, Joel
From: tsc@... <tsc@...>
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
> [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. > I am not 100% certain it is still a FD.io project?
TRex is still FD.io project as far as I know. Git+Gerrit server at gerrit.fd.io is just a tool, projects are not required to use it.
The github repo is the official primary one for TRex, while CSIT and VPP still use gerrit.fd.io as the primary, and github just as a mirror.
> counts as distributing the software.
TRex offers releases for download, for example [19]. They are basically archives of git content at that time, including the problematic Python bits.
> Can we be clear that anything in CSIT that links/loads > scapy directly is GPL’ed (or a compatible license).
Looking at the Charter [18], we can do that according to 7.c, (giving us the exception to otherwise prohibiting 7.b.i and 7.b.iii). The downside is, around 90% or more of CSIT code would end up as GPL (almost everything is linked together).
Reducing the percentage by offloading the scapy processing into a separate process is of course possible, but it would also require a considerable effort to finish.
Anyway, switching parts of code to GPL quickly and then reducing the amount of code affected slowly looks like a good plan to me. 7.c just requires a request, justification, and two-thirds TSC vote.
Vratko.
[19] https://github.com/cisco-system-traffic-generator/trex-core/releases/tag/v2.61
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> Why is TRex’s licensing being questioned?
It just happened to be the first FD.io project examined (by Miroslav) for dependencies. VPP and CSIT were examined (by Vratko) few days later. [rk] Ok my understand is that TRex had moved to github – that TRex repo is dead. I am not 100% certain it is still a FD.io project?
> If we don’t redistribute, can’t we just take it’s licensing at face value?
We (CSIT) could do that, except that CSIT also uses scapy directly. [rk] Ok, can we focus and resolve on CSIT’s usage first [rk] Can we be clear that anything in CSIT that links/loads scapy directly is GPL’ed (or a compatible license). [rk] Anything else (non-GPL’ed) that talk’s to scapy via interfaces that preserve isolation?
We (FD.io) cannot do that, as TRex is a FD.io project which distributes TRex code, including the problematic Python bits. [rk] Ok – we need Ed to check this. [rk] In that case, should it not be someone from TRex covering it?
I believe even having the git repository publicly cloneable counts as distributing the software. [rk] Perhaps, but there has been no contributions to the FD.io VPP TRex git since 2017 [rk] We need to ask, what the best approach is here.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Ok … we need to contain the problem domain here, as it is way too complicated at the moment.
Why is TRex’s licensing being questioned? Aren’t we just consumers or TRex, we don’t redistribute do we (is it still a FD.io project). If we don’t redistribute, can’t we just take it’s licensing at face value?
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
>> I am not entirely sure how vpp "make test" sends packets to VPP. > [rk] I thought it was veth network interface or similar, can you check?
It seems this document [11] still applies. The python test code uses the linked scapy to define packet objects, turns those (by calling a scapy function) into a pcap file, and notifies a (separate) VPP process to read the file.
Receiving is the inverse. VPP creates pcap files, and python test code parses that (once again using linked scapy). [rk] Ok – this appears to be safe then.
>> Scapy does not send packets to TRex, scapy objects are only used >> as a "template" for TRex to create packets (in C) from. > [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?
A specific "template" is defined in CSIT repository (as a set of .py files, for example [14], already using scapy classes). During test setup, the files are copied to the machine that has TRex installed. When test starts, another CSIT python utility (not directly linked with scapy) is started on the TRex machine, it imports and calls [13] the python part of TRex installation, which dynamically imports [12] the template in question. In the previous e-mail I mentioned this [5] file, part of TRex python code, which further processes the template, occasionally using direct calls to scapy functions.
I am not sure how the information processing continues towards the C engine of TRex, but the description so far should be enough to show the license issues of the current usage.
TRex also accepts different formats (json, yaml and pcap) for the "templates". Not sure whether their processing also relies on scapy, but CSIT does not use those formats.
> We need an IP diagram with the modules/processes involved, > how there are linked and their licensing.
Usually, there is only one process with licensing issues (running Python interpreter), but with way too many modules for me to track.
Or did you mean python packages (instead of modules)? Direct dependencies are [15] for main CSIT and [16] for VPP test. I am not sure what the dependencies are for TRex, but [17] seems to be a good list.
Vratko.
[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf [13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112 [15] https://github.com/FDio/csit/blob/master/requirements.txt [16] https://github.com/FDio/vpp/blob/master/test/requirements.txt [17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs
From: tsc@... <tsc@...>
On Behalf Of Ray Kinsella
Hi Vratko,
Inline.
From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
The setup details differ. The C projects (VPP and the main part of TRex server) are usually separate processes, being accessed by official interfaces. But each setup contains (at least) one process which is running Python interpreter running mixed code. [rk] Ok, the principle of separation is important here. [rk] As anything that ‘links’ directly with Scapy is affected. [rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc. By mixed code I mean the upper layers are coming from FD.io repositories (thus Apache license 2.0 applies), but the virtualenv contains scapy installation. [rk] Containing a Scapy installation does not automatically effect anything else. [rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’. The upper layers are importing scapy modules and classes, instantiating scapy objects and calling scapy methods. This would be permitted if scapy was LGPL, but not for GPL. [rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.
[rk] For TRex/VPP [rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing. [rk] It doesn’t need to be exhaustive, but it needs to show the main actors. [rk] Ed – is there a LF process to resolve these kinds of issues?
>
How is the data (packets) scapy creates consumed by VPP/TRex,
[rk] This is safe, I would think. I am not entirely sure how vpp "make test" sends packets to VPP. [rk] I thought it was veth network interface or similar, can you check? Scapy does not send packets to TRex, scapy objects are only used as a "template" for TRex to create packets (in C) from. [rk] Ok – but we need detail here, how does TRex – ‘get’ the template? In VPP and CSIT tests, scapy is mainly used to parse packets received (via an official interface) from VPP. [rk] Understood
> how TRex & VPP “talk” to scapy
Simplified from the above: TRex and VPP (almost) do not talk to scapy on their own. But most verify jobs are running fd.io python programs which depend on scapy as a library.
Vratko.
From: Kinsella, Ray <ray.kinsella@...>
Hi Vratko,
So there is a lot of information thanks for putting this together. What is of primary importance to understand is how TRex & VPP “talk” to scapy.
Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.
Some questions
Thanks,
Ray K
From: tsc@... [mailto:tsc@...]
On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Adding some technical details with quick links.
Scapy license [0] is GLP, not LGPL.
In python, using "import" statement amounts to linking, which creates [1] a combined program, instead of a mere aggregation of different programs.
FD.io projects apply Apache 2.0 license, TRex [2], VPP [3] and CSIT [4].
The scapy code is imported, and the internal data structures are used by TRex [5], VPP [6] and CSIT python code
(source code is in C), and the packaged Python code is not related to scapy as far as I know. which run the code containing (modified) scapy parts and original VPP python parts linked together,
[0] https://github.com/secdev/scapy/blob/master/LICENSE [1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation [2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE [3] https://github.com/FDio/vpp/blob/master/LICENSE [4] https://github.com/FDio/csit/blob/master/LICENSE [5]
https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13 [7]
https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24 [9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122 [10] https://www.apache.org/licenses/GPL-compatibility.html
From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer. Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).
Ed, Ray,
Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps. Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?
Cheers, -Maciek
|
|
|