Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]

Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

>> I am not entirely sure how vpp "make test" sends packets to VPP.

> [rk] I thought it was veth network interface or similar, can you check?

 

It seems this document [11] still applies.

The python test code uses the linked scapy to define packet objects,

turns those (by calling a scapy function) into a pcap file,

and notifies a (separate) VPP process to read the file.

 

Receiving is the inverse. VPP creates pcap files,

and python test code parses that (once again using linked scapy).

 

>> Scapy does not send packets to TRex, scapy objects are only used

>> as a "template" for TRex to create packets (in C) from.

> [rk] Ok – but we need detail here, how does TRex – ‘get’ the template?

 

A specific "template" is defined in CSIT repository

(as a set of .py files, for example [14], already using scapy classes).

During test setup, the files are copied to the

machine that has TRex installed.

When test starts, another CSIT python utility

(not directly linked with scapy) is started on the TRex machine,

it imports and calls [13] the python part of TRex installation,

which dynamically imports [12] the template in question.

In the previous e-mail I mentioned this [5] file,

part of TRex python code, which further processes the template,

occasionally using direct calls to scapy functions.

 

I am not sure how the information processing

continues towards the C engine of TRex,

but the description so far should be enough

to show the license issues of the current usage.

 

TRex also accepts different formats (json, yaml and pcap)

for the "templates". Not sure whether their processing

also relies on scapy, but CSIT does not use those formats.

 

> We need an IP diagram with the modules/processes involved,

> how there are linked and their licensing.

 

Usually, there is only one process with licensing issues

(running Python interpreter), but with way too many modules

for me to track.

 

Or did you mean python packages (instead of modules)?

Direct dependencies are [15] for main CSIT and [16] for VPP test.

I am not sure what the dependencies are for TRex,

but [17] seems to be a good list.

 

Vratko.

 

[11] https://github.com/FDio/vpp/blob/master/test/doc/overview.rst#packet-flow-in-the-vtf

[12] https://github.com/cisco-system-traffic-generator/trex-core/blob/v2.61/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_streams.py#L1029

[13] https://github.com/FDio/csit/blob/master/resources/tools/trex/trex_stateless_profile.py#L111-L112

[14] https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/trex-sl-2n-dot1qip4asym-ip4src254.py

[15] https://github.com/FDio/csit/blob/master/requirements.txt

[16] https://github.com/FDio/vpp/blob/master/test/requirements.txt

[17] https://github.com/cisco-system-traffic-generator/trex-core/tree/v2.61/scripts/external_libs

 

From: tsc@... <tsc@...> On Behalf Of Ray Kinsella
Sent: Monday, October 14, 2019 10:57 AM
To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...>
Subject: Re: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]

 

Hi Vratko,

 

Inline.

 

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) [mailto:vrpolak@...]
Sent: Friday 11 October 2019 17:20
To: Kinsella, Ray <ray.kinsella@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...>
Subject: RE: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]

 

> How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?

 

The setup details differ. The C projects (VPP and the main part of TRex server)

are usually separate processes, being accessed by official interfaces.

But each setup contains (at least) one process

which is running Python interpreter running mixed code.

[rk] Ok, the principle of separation is important here.

[rk] As anything that ‘links’ directly with Scapy is affected.

[rk] You _can_ still talk to Scapy via an interface that achieves separation; stdin/stdout, network, socket etc.

By mixed code I mean the upper layers are coming from FD.io repositories

(thus Apache license 2.0 applies), but the virtualenv contains scapy installation.

[rk] Containing a Scapy installation does not automatically effect anything else.

[rk] You are, however obliged to ensure the source code of the version of scapy you used was ‘available on request’.

The upper layers are importing scapy modules and classes,

instantiating scapy objects and calling scapy methods.

This would be permitted if scapy was LGPL, but not for GPL.

[rk] Agreed anything that ‘link’s with Scapy in this way, would have to be GPL’ed I imagine.

 

[rk] For TRex/VPP

[rk] We need an IP diagram with the modules/processes involved, how there are linked and their licensing.

[rk] It doesn’t need to be exhaustive, but it needs to show the main actors.

[rk] Ed – is there a LF process to resolve these kinds of issues?

 

> How is the data (packets) scapy creates consumed by VPP/TRex,
> through an API or a Unix Socket or a Network Interface?


For CSIT usage, VPP consumes packets only via network interface.

[rk] This is safe, I would think.

I am not entirely sure how vpp "make test" sends packets to VPP.

[rk] I thought it was veth network interface or similar, can you check?

Scapy does not send packets to TRex, scapy objects are only used

as a "template" for TRex to create packets (in C) from.

[rk] Ok – but we need detail here, how does TRex – ‘get’ the template?

In VPP and CSIT tests, scapy is mainly used

to parse packets received (via an official interface) from VPP.

[rk] Understood

 

> how TRex & VPP “talk” to scapy

 

Simplified from the above:

TRex and VPP (almost) do not talk to scapy on their own.

But most verify jobs are running fd.io python programs

which depend on scapy as a library.

 

Vratko.

 

From: Kinsella, Ray <ray.kinsella@...>
Sent: Friday, October 11, 2019 10:58 AM
To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>; Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...>
Subject: RE: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]

 

Hi Vratko,

 

So there is a lot of information thanks for putting this together.

What is of primary importance to understand is how TRex & VPP “talk” to scapy.

 

Scapy is GPL, any modifications made to Scapy are therefore clearly covered by the GPL.

 

Some questions

 

  1. How is scapy invoked, is it a separate process or is it directly like with VPP/TRex?
  2. How is the data (packets) scapy creates consumed by VPP/TRex,
    through an API or a Unix Socket or a Network Interface?

 

Thanks,

 

Ray K

 

From: tsc@... [mailto:tsc@...] On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Sent: Thursday 10 October 2019 17:24
To: Maciek Konstantynowicz (mkonstan) <mkonstan@...>; tsc@...; Ed Warnicke (eaw) <eaw@...>; Kinsella, Ray <ray.kinsella@...>
Cc: tsc@...
Subject: Re: [tsc] Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]

 

Adding some technical details with quick links.

 

Scapy license [0] is GLP, not LGPL.

 

In python, using "import" statement amounts to linking,

which creates [1] a combined program, instead of

a mere aggregation of different programs.

 

FD.io projects apply Apache 2.0 license,
globally to the whole repository in case of

TRex [2], VPP [3] and CSIT [4].

 

The scapy code is imported, and the internal data structures

are used by TRex [5], VPP [6] and CSIT python code
([7] indirectly via TRex for performance tests,
[8] directly for functional tests).
Additionally, VPP even patches [9] the scapy code,
so it uses a modified code.


The product of VPP is packaged mostly as binary

(source code is in C), and the packaged Python code

is not related to scapy as far as I know.

But VPP verify jobs are executing tests,

which run the code containing (modified) scapy parts

and original VPP python parts linked together,
which is not allowed by GLP2 (as far as I understand).

Similarly, CSIT jobs are running performance and functional tests
with python parts linking together incompatible [10] licenses.


Vratko.

[0] https://github.com/secdev/scapy/blob/master/LICENSE

[1] https://www.gnu.org/licenses/gpl-faq.en.html#MereAggregation

[2] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/LICENSE

[3] https://github.com/FDio/vpp/blob/master/LICENSE

[4] https://github.com/FDio/csit/blob/master/LICENSE

[5] https://github.com/cisco-system-traffic-generator/trex-core/blob/master/scripts/automation/trex_control_plane/interactive/trex/stl/trex_stl_packet_builder_scapy.py#L13
[6] https://github.com/FDio/vpp/blob/master/test/test_ip4.py#L7-L11

[7] https://github.com/FDio/csit/blob/master/resources/traffic_profiles/trex/profile_trex_stateless_base_class.py#L24
[8] https://github.com/FDio/csit/blob/master/resources/libraries/python/PacketVerifier.py#L70-L73

[9] https://github.com/FDio/vpp/blob/master/test/Makefile#L120-L122

[10] https://www.apache.org/licenses/GPL-compatibility.html

 

From: Maciek Konstantynowicz (mkonstan) <mkonstan@...>
Sent: Thursday, October 10, 2019 5:47 PM
To: tsc@...; Ed Warnicke (eaw) <eaw@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Miroslav Los -X (mirlos - PANTHEON TECHNOLOGIES at Cisco) <mirlos@...>; Hanoch Haim (hhaim) <hhaim@...>; Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) <vrpolak@...>
Subject: Re: Scapy licensing issue in FD.io projects [Was: TRex distribution and licensing of external libraries]

 

Resending from Cisco email account as it’s back and cleared for sending emails to tsc mailer.

Adding Miroslav, Hanoh and Vratko who has been also involved here (Miroslav brought up the issue).

 

Ed, Ray,

 

Per our discussion on FD.io TSC call few mins ago, pls advise on the next steps.

Specifically, how to articulate this case in a crisp and succinct manner for LF(N) legal advisor?

 

Cheers,

-Maciek

 

On 10 Oct 2019, at 16:15, Maciek Konstantynowicz <mackonstan@...> wrote:

 

Correcting the email subject to make it clear it’s related to Scapy licensing agenda topic for today’s FD.io TSC meeting.

 

Cheers,
-Maciek

 

On 10 Oct 2019, at 16:12, Maciek Konstantynowicz <mackonstan@...> wrote:

 

Hi, Here is the background to Scapy licensing agenda topic for today’s FD.io TSC meeting.

 

Cheers,
-Maciek

 

Begin forwarded message:

 

From: "Miroslav Los -X (mirlos - PANTHEON TECHNOLOGIES at Cisco)" <mirlos@...>

Subject: TRex distribution and licensing of external libraries

Date: 26 September 2019 at 15:02:36 BST

To: "Hanoch Haim (hhaim)" <hhaim@...>

Cc: "Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)" <vrpolak@...>, "Maciek Konstantynowicz (mkonstan)" <mkonstan@...>

 

Hi,

 

I’ve been looking at the dependencies for TRex’s client library, in order to find if it could be distributed independent of the complete trex-core distribution.

 

I have noticed that the external_lib contains code from several other projects, and may or may not have been modified.

It is reportedly the case for at least pyzmq-ctypes and scapy. Pyzmq-ctypes, apparently abandoned by its original authors, is licensed under GNU LGPL.

 

Whereas scapy is GPLv2 – in the 2.3.1 version included, this was indicated in the setup.py file of the original release sources on github; the authors added an explicit LICENSE file with GPLv2 for the next version. Nonetheless, TRex would not have any license other than the GPLv2 which would let it distribute scapy 2.3.1.

 

I do not see the licensing terms for either of those packages indicated in the distribution, or repository. Nor do I believe the Apache 2.0 license of TRex can be applied to that code.

 

And most importantly, any code that imports those modules, as it may be a derived work. I’ve only seen scapy used by code under the scripts directory. I think the licensing of those parts of TRex distribution should be clarified, or otherwise resolved. Thoughts?

 

Thanks,

Miroslav

 

 

 

Join tsc@lists.fd.io to automatically receive all group messages.