[csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io


Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave,

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 


Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

> Just like VPP:

>> https://packagecloud.io/fdio/master

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

>> https://packagecloud.io/fdio/vsap/master

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

@Dave Wallace

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

[2] https://packagecloud.io/fdio

[3] https://github.com/FDio/ci-management/blob/be612c3280a8e8e7a67431bf2f8aa55c81b255cb/jjb/scripts/packagecloud_push.sh#L2

 

From: Jiang, XiaolongX <xiaolongx.jiang@...>
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

>https://packagecloud.io/fdio/master

 

VSAP maybe like this:

>https://packagecloud.io/fdio/vsap/master

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave,

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 


Pei, Yulong
 

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,



On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?



The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?



The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a



 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a




On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 


Dave Wallace
 

Ping,

The fundamental problem that I see with this proposal is who supports the VSAP release. 
The VPP community does not have the capacity nor the charter to support use-case specific forks of VPP.
Then there is the problem of the precedent set of VPP supporting project specific VPP releases which devolves into a rats nest of variants.

For me, this is a non-starter.

Thanks,
-daw-


On 6/2/2020 11:04 PM, Yu, Ping wrote:

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 



On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,




On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?




The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?




The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a




 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a





On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 



Dave Wallace
 

Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,



On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?



The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?



The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a



 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a




On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 



    


Pei, Yulong
 

Hi Dave,

 

Thanks for your comments,

 

My answers inline,

 

 

 

From: Dave Wallace <dwallacelf@...>
Sent: Wednesday, June 3, 2020 9:25 PM
To: Pei, Yulong <yulong.pei@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

 

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.


Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.


Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

 

Best Regards

Yulong Pei


Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,




On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?




The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?




The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a




 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a





On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 




 


Pei, Yulong
 

Hi Guys,

 

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

 

Best Regards

Yulong Pei

 

From: Pei, Yulong
Sent: Monday, June 8, 2020 11:31 AM
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <XiaolongX.Jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Dave,

 

Thanks for your comments,

 

My answers inline,

 

 

 

From: Dave Wallace <dwallacelf@...>
Sent: Wednesday, June 3, 2020 9:25 PM
To: Pei, Yulong <yulong.pei@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

 

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.


Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.


Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

 

Best Regards

Yulong Pei


Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,



On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?



The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?



The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a



 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a




On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 

 


 


Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
 

I assume readers are familiar with most of the context

discussed in other communication channels.

If not, I recommend to take a peek at the long discussion

on "fdio-infra" channel of fdio slack,

starting from June 1st.

 

After some thinking, I have realized ci-management

is already conceptually prepared to handle

multiple "areas" of compatibility,

where artifacts within the same area are compatible,

but artifacts from different areas are not compatible.

 

For snapshot artifacts, the compatibility is assumed

only for the latest uploaded versions.

Temporary breakages due to API changes are tolerated.

Because of that, the concept is not named area,

but it is named "stream".

 

It is an abstraction, evolved from the concept of branch name;

needed for integration of sub-projects whose branch names do not match exactly.

(In fd.io ci-management, packagecloud repository name

matches the stream exactly, but it is not a requirement.

For example, when we used Nexus, repository name did not match directly.)

 

Anyway, in this sense, master package repository [0]

is for VPP artifacts built from VPP master branch,

and any other artifacts compatible with (latest version of) them.

VPP code in stable/2005 branch creates snapshot artifacts

uploaded to 2005 package repository [1].

 

If VSAP needs VPP builds different from both master and 2005,

it means new stream needs to be created, for example called "vsap".

VSAP merge job for vsap stream can use VSAP master branch,

and there will be no VSAP merge job for master stream.

 

In future, if VPP project accepts all VSAP-related settings,

(and VPP verify job ensures production APIs are never changed)

the following process can be applied.

A new VSAP branch (called "vsap") is cut from master branch,

and the VSAP merge job for stream vsap starts using that branch.

In VSAP master branch, the build scripts are changed

to no longer build VPP artifacts (maybe download but delete before pushing).

Then a VSAP merge job for stream master is created,

using VSAP master branch, and uploading VSAP (but not VPP) artifacts to [0].

For a period of time, both streams will be tested,

to confirm that VSAP artifacts from stream master

work equally well than VSAP (and VPP) artifacts from stream vsap.

Then, vsap branch can stop being updated and all vsap stream jobs can be deleted.

 

In other words, the "vsap" stream acts as a "feature branch" for VPP project,

except that it is maintained by VSAP (not VPP) developers.

This is more flexible than my previous idea of giving

each sub-project their own packagecloud repository.

Firstly, work can be done on multiple features.

For example streams "vsap_speed" and "vsap_scale" can at first

hardcode a particular "speed versus memory" tradeoffs,

each can be upstreamed to VPP at a different time.

Secondly, when we have more cooperating sub-projects,

they can push their artifacts into the same repository,

so that users (and testers) have easier time installing them.

 

There was some discussion on TSC meeting,

mostly related to hICN sub-project instead.

As usual, I am trying to find a solution suitable for both hICN and VSAP.

For hICN, the stream solution is to mark their merge job

(which uses hICN master branch) as stream 2005.

That assumes 2005 VPP snapshots are stable enough for them.

 

What do you think?

Any important question that does not fit into the stream abstraction?

 

Vratko.

 

[0] https://packagecloud.io/fdio/master

[1] https://packagecloud.io/fdio/2005

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Tuesday, 2020-June-09 11:20
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io
Importance: High

 

Hi Guys,

 

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

 

Best Regards

Yulong Pei

 

From: Pei, Yulong
Sent: Monday, June 8, 2020 11:31 AM
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <XiaolongX.Jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Dave,

 

Thanks for your comments,

 

My answers inline,

 

 

 

From: Dave Wallace <dwallacelf@...>
Sent: Wednesday, June 3, 2020 9:25 PM
To: Pei, Yulong <yulong.pei@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

 

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.


Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.


Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

 

Best Regards

Yulong Pei


Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

 

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

 

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

 

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a



On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 

 

 

 


Pei, Yulong
 

Hi,

 

I agree  Vratko’s stream abstraction solution,  if no other objection,

 

1. who can help to set up  https://packagecloud.io/fdio/vsap  repository ?

 

2. how to get permission  to manage to run  ` package_cloud push fdio/vsap  /path/to/local/vsap/package` ?

 

Best Regards

Yulong Pei

 

From: csit-dev@... <csit-dev@...> On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Friday, June 12, 2020 6:42 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

I assume readers are familiar with most of the context

discussed in other communication channels.

If not, I recommend to take a peek at the long discussion

on "fdio-infra" channel of fdio slack,

starting from June 1st.

 

After some thinking, I have realized ci-management

is already conceptually prepared to handle

multiple "areas" of compatibility,

where artifacts within the same area are compatible,

but artifacts from different areas are not compatible.

 

For snapshot artifacts, the compatibility is assumed

only for the latest uploaded versions.

Temporary breakages due to API changes are tolerated.

Because of that, the concept is not named area,

but it is named "stream".

 

It is an abstraction, evolved from the concept of branch name;

needed for integration of sub-projects whose branch names do not match exactly.

(In fd.io ci-management, packagecloud repository name

matches the stream exactly, but it is not a requirement.

For example, when we used Nexus, repository name did not match directly.)

 

Anyway, in this sense, master package repository [0]

is for VPP artifacts built from VPP master branch,

and any other artifacts compatible with (latest version of) them.

VPP code in stable/2005 branch creates snapshot artifacts

uploaded to 2005 package repository [1].

 

If VSAP needs VPP builds different from both master and 2005,

it means new stream needs to be created, for example called "vsap".

VSAP merge job for vsap stream can use VSAP master branch,

and there will be no VSAP merge job for master stream.

 

In future, if VPP project accepts all VSAP-related settings,

(and VPP verify job ensures production APIs are never changed)

the following process can be applied.

A new VSAP branch (called "vsap") is cut from master branch,

and the VSAP merge job for stream vsap starts using that branch.

In VSAP master branch, the build scripts are changed

to no longer build VPP artifacts (maybe download but delete before pushing).

Then a VSAP merge job for stream master is created,

using VSAP master branch, and uploading VSAP (but not VPP) artifacts to [0].

For a period of time, both streams will be tested,

to confirm that VSAP artifacts from stream master

work equally well than VSAP (and VPP) artifacts from stream vsap.

Then, vsap branch can stop being updated and all vsap stream jobs can be deleted.

 

In other words, the "vsap" stream acts as a "feature branch" for VPP project,

except that it is maintained by VSAP (not VPP) developers.

This is more flexible than my previous idea of giving

each sub-project their own packagecloud repository.

Firstly, work can be done on multiple features.

For example streams "vsap_speed" and "vsap_scale" can at first

hardcode a particular "speed versus memory" tradeoffs,

each can be upstreamed to VPP at a different time.

Secondly, when we have more cooperating sub-projects,

they can push their artifacts into the same repository,

so that users (and testers) have easier time installing them.

 

There was some discussion on TSC meeting,

mostly related to hICN sub-project instead.

As usual, I am trying to find a solution suitable for both hICN and VSAP.

For hICN, the stream solution is to mark their merge job

(which uses hICN master branch) as stream 2005.

That assumes 2005 VPP snapshots are stable enough for them.

 

What do you think?

Any important question that does not fit into the stream abstraction?

 

Vratko.

 

[0] https://packagecloud.io/fdio/master

[1] https://packagecloud.io/fdio/2005

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Tuesday, 2020-June-09 11:20
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io
Importance: High

 

Hi Guys,

 

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

 

Best Regards

Yulong Pei

 

From: Pei, Yulong
Sent: Monday, June 8, 2020 11:31 AM
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <XiaolongX.Jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Dave,

 

Thanks for your comments,

 

My answers inline,

 

 

 

From: Dave Wallace <dwallacelf@...>
Sent: Wednesday, June 3, 2020 9:25 PM
To: Pei, Yulong <yulong.pei@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

 

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.


Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.


Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

 

Best Regards

Yulong Pei


Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

 

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

 

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

 

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a

 

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 

 

 

 


Dave Wallace
 

Ping,

Additional comment inline...


On 6/22/20 9:59 PM, Florin Coras wrote:
Hi Ping, 

On Jun 22, 2020, at 6:48 PM, Yu, Ping <ping.yu@...> wrote:

Hi, Florin,
 
After testing, we found that if without VPP patch, the performance will dropped too much, thus it is not meaningfully to integrate VPP master only and provide the CSIT result, which will disappointed people.

Approximately what type of difference are we talking about? I guess this is mainly due to the way accepted connections are load balanced across vcl workers? 

At the other hand, we’d like to integrate with OpenSSL 3.0.0 alpha release even without changing code in VPP master branch, so this is definitely different from sanity VPP.

Okay, that’s a different type of problem, unfortunately, and that does indeed need different binaries. 

Based on above problem, we’d like to separate VSAP VPP build from native VPP build.   Does it make sense?
We will look forward to the VSAP repo based build, and before this repo comes out, we’d like to change the vpp build name to avoid conflict.
There is an additional complication here -- who supports the forked VSAP version of VPP?

This issue was raised during the VSAP approval by the TSC who strongly objected to having a publicly available fork of VPP as part of the VSAP project.  It is acceptable to have experimental contributions to VPP as part of the VPP project (e.g. quic plugin). IMHO any modifications made to enhance VSAP performance test results must be part of the VPP project for the results to be meaningful.

Anyone can fork an open source project, modify it for a specific use case while in the process making changes which are unacceptable to the project maintainers.  In that case whomever creates the fork is responsible for supporting the entire forked repository.  

VSAP is an official FD.io project, thus any modifications to VPP are by association owned by the VPP maintainers.  Which means the VPP code used by VSAP must have been reviewed, approved, and merged by the VPP committers.  If you insist on changing the charter of the VSAP project, then I would recommend that we bring this proposal to the TSC for approval.

Thanks,
-daw-

That’s fine with me, but ultimately the CSIT maintainers need to agree to this :-)

Cheers,
Florin

 
Ping
 
From: Florin Coras <fcoras.lists@...> 
Sent: Monday, June 22, 2020 10:08 PM
To: Yu, Ping <ping.yu@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; vrpolak@...; Dave Wallace <dwallacelf@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io
 
What do you mean by that, Ping? 
 
Florin


On Jun 22, 2020, at 1:42 AM, Yu, Ping <ping.yu@...> wrote:
 
Hello, all
 
I once talked with Florin that VSAP can temporarily use un-patch version, but it seems the performance is very poor, so it will not show any performance for VSAP. We’d like to consider Vratko’s solution.
Before the vsap repository comes out, we can replace vpp packet name to be vsap_vpp to avoid conflict with vpp master branch.
 
Ping
 
From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, June 22, 2020 10:09 AM
To: Pei, Yulong <yulong.pei@...>; vrpolak@...; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
I agree with Vratko, too. 
VSAP greatly improves the performance of Nginx, which may be helpful for the promotion of VPP.
 
Best Regards
Xiaolong Jiang
 
From: Pei, Yulong <yulong.pei@...> 
Sent: Thursday, June 18, 2020 2:06 PM
To: vrpolak@...; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
Hi,
 
I agree  Vratko’s stream abstraction solution,  if no other objection,
 
1. who can help to set up  https://packagecloud.io/fdio/vsap  repository ? 
 
2. how to get permission  to manage to run  ` package_cloud push fdio/vsap  /path/to/local/vsap/package` ?
 
Best Regards
Yulong Pei
 
From: csit-dev@... <csit-dev@...> On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Friday, June 12, 2020 6:42 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
I assume readers are familiar with most of the context
discussed in other communication channels.
If not, I recommend to take a peek at the long discussion
on "fdio-infra" channel of fdio slack,
starting from June 1st.
 
After some thinking, I have realized ci-management
is already conceptually prepared to handle
multiple "areas" of compatibility,
where artifacts within the same area are compatible,
but artifacts from different areas are not compatible.
 
For snapshot artifacts, the compatibility is assumed
only for the latest uploaded versions.
Temporary breakages due to API changes are tolerated.
Because of that, the concept is not named area,
but it is named "stream".
 
It is an abstraction, evolved from the concept of branch name;
needed for integration of sub-projects whose branch names do not match exactly.
(In fd.io ci-management, packagecloud repository name
matches the stream exactly, but it is not a requirement.
For example, when we used Nexus, repository name did not match directly.)
 
Anyway, in this sense, master package repository [0]
is for VPP artifacts built from VPP master branch,
and any other artifacts compatible with (latest version of) them.
VPP code in stable/2005 branch creates snapshot artifacts
uploaded to 2005 package repository [1].
 
If VSAP needs VPP builds different from both master and 2005,
it means new stream needs to be created, for example called "vsap".
VSAP merge job for vsap stream can use VSAP master branch,
and there will be no VSAP merge job for master stream.
 
In future, if VPP project accepts all VSAP-related settings,
(and VPP verify job ensures production APIs are never changed)
the following process can be applied.
A new VSAP branch (called "vsap") is cut from master branch,
and the VSAP merge job for stream vsap starts using that branch.
In VSAP master branch, the build scripts are changed
to no longer build VPP artifacts (maybe download but delete before pushing).
Then a VSAP merge job for stream master is created,
using VSAP master branch, and uploading VSAP (but not VPP) artifacts to [0].
For a period of time, both streams will be tested,
to confirm that VSAP artifacts from stream master
work equally well than VSAP (and VPP) artifacts from stream vsap.
Then, vsap branch can stop being updated and all vsap stream jobs can be deleted.
 
In other words, the "vsap" stream acts as a "feature branch" for VPP project,
except that it is maintained by VSAP (not VPP) developers.
This is more flexible than my previous idea of giving
each sub-project their own packagecloud repository.
Firstly, work can be done on multiple features.
For example streams "vsap_speed" and "vsap_scale" can at first
hardcode a particular "speed versus memory" tradeoffs,
each can be upstreamed to VPP at a different time.
Secondly, when we have more cooperating sub-projects,
they can push their artifacts into the same repository,
so that users (and testers) have easier time installing them.
 
There was some discussion on TSC meeting,
mostly related to hICN sub-project instead.
As usual, I am trying to find a solution suitable for both hICN and VSAP.
For hICN, the stream solution is to mark their merge job
(which uses hICN master branch) as stream 2005.
That assumes 2005 VPP snapshots are stable enough for them.
 
What do you think?
Any important question that does not fit into the stream abstraction?
 
Vratko.
 
 
From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Tuesday, 2020-June-09 11:20
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
Importance: High
 
Hi Guys,
 
If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.
 
Best Regards
Yulong Pei
 
From: Pei, Yulong 
Sent: Monday, June 8, 2020 11:31 AM
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <XiaolongX.Jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
Hi Dave,
 
Thanks for your comments,
 
My answers inline,
 
 
 
From: Dave Wallace <dwallacelf@...> 
Sent: Wednesday, June 3, 2020 9:25 PM
To: Pei, Yulong <yulong.pei@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:
Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,
This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.
 
When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.
Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages.  
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 
 
[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.


Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test, 
As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.


Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date. 

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.
 
Best Regards
Yulong Pei


Thanks,
-daw-

 
Best Regards
Yulong Pei
 
From: Yu, Ping <ping.yu@...> 
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch)<ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io
 
Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first. 
 
@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.
 
Ping 
 
From: Florin Coras <fcoras.lists@...> 
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io
 
Hi Ping, 
 
I guess there are two improvements that have not entirely made their way into vpp:
- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 
- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 
 
To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 
 
Regards,
Florin
 
 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:
 
Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.
 
For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  
 
 
From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io
 
Hi Ping,

 

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 
Hi, Andrew
 
You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.
 
The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

 

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.
 
Could you try to outline the considerations or give a pointer where I can read up?
 
I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  
 
Do you add a custom repository for each package that you install on your system ?

 

The solution is not difficult, and just provide repo for each project.
 
Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 
 
This needs to be discussed at TSC I think. 
 
Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 
 
This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?
 
—a

 

 
Ping
 
From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
 
1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 
 
2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.
 
What are the obstacles that prevent that ?
 
--a

 

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 
> Just like VPP:
 
Currently, it is a "fdio" repository,
projects other than VPP are uploading there too.
 
> In order to avoid confusion, we would like to have a new packagecloud repository.
> VSAP maybe like this:
 
So this is a different pattern,
but I agree it is probably the simplest solution
for allowing different project to upload packages of the same name.
Later you will probably also need /vsap/release repo.
 
Looking at [2], there already are several repositories
unrelated to STREAM variable.
So creating vsap/master could work,
unless packagecloud limits the repository level.
In that case we could create something like vsap-master.
 
The main issue will be in the merge job,
as packagecloud_push.sh directly uses PCIO_CO [3] env var.
You would need to introduce a new environment variable
(to pass "vsap" while keeping STREAM for "master")
and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}
(without affecting jobs that do not set the new environment variable).
 
What do you think?
Should we start creating per-project repositories,
or should we keep trying to reuse fdio/master?
I personally prefer per-project repositories,
just so I do not have to investigate console output
of every new project verify/merge job. :)
 
I think it would also help with HICN release frequency,
if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.
 
Vratko.
 
 
From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
> When installing the nginx .deb package,
> does it contain statically linked parts of VPP it needs,
> or does a user need to also install the patched vpp .deb packages?
 
The latter.
 
 
> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?
 
In order to avoid confusion, we would like to have a new packagecloud repository.
Just like VPP:
 
VSAP maybe like this:
 
 
Best Regards
Xiaolong Jiang
 
From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
Before we enable VSAP merge job,
I would like to understand the build process more.
 
So I looked at verify job console output.
(Link [0] will stop working when Sandbox is cleared over weekend.)
 
Observations (and some comments) first:
 
 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10
 
The job was triggered for VSAP master head. Ok.
 
 > git config --get submodule.vpp.url # timeout=10
 
Vpp repository is acts as a submodule for VSAP. Ok.
 
--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log
 
Openssl is built, I assume there is a good reason to do that.
 
Applying patch: 0001-3.0.0.patch
patching file src/plugins/crypto_openssl/CMakeLists.txt
 
So VSAP does not use "vanilla" VPP,
but applies some patches.
That is normal during the development,
VSAP working with the patched VPP
acts as a kind of indirect verify job
for upstreaming the patch into VPP repo.
 
dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.
 
Note that packagecloud already contains a vpp ~73 .deb [1],
it comes from VPP merge job.
The dangerous part is that even the ge part is the same,
despite the patches being applied here.
Only the ~b part happens to be different,
but in production also that could happen to be the same.
 
{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}
 
I believe this is the main output the VSAP merge job will upload to packagecloud.
 
Now the main question.
 
When installing the nginx .deb package,
does it contain statically linked parts of VPP it needs,
or does a user need to also install the patched vpp .deb packages?
 
If the former, the VSAP build process should make sure
any unneeded .deb files are deleted (or better yet not even created)
before proceeding to packagecloud upload.
 
If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?
Uploading them to the same packagecloud repository will certainly confuse users.
One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.
Short term by patching VPP makefiles,
longer-term by upstreaming a Change that takes prefix from an environment variable.
 
Vratko.
 
 
From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
 
Forgot to add Ping & Xiaolong in the email To list.
 
From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package topackagecloud.io and download package from packagecloud.io
 
Hello Dave, 
 
Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?
 
Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,
 
 
Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?
 
And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?
 
Best Regards
Yulong Pei
 
 

 

 
 



Pei, Yulong
 

Hi Florin & Ping,

 

My comment about vpp built with openssl-3.0.0,

See inline,

 

From: csit-dev@... <csit-dev@...> On Behalf Of Florin Coras
Sent: Tuesday, June 23, 2020 9:59 AM
To: Yu, Ping <ping.yu@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; vrpolak@...; Dave Wallace <dwallacelf@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 



On Jun 22, 2020, at 6:48 PM, Yu, Ping <ping.yu@...> wrote:

 

Hi, Florin,

 

After testing, we found that if without VPP patch, the performance will dropped too much, thus it is not meaningfully to integrate VPP master only and provide the CSIT result, which will disappointed people.

 

Approximately what type of difference are we talking about? I guess this is mainly due to the way accepted connections are load balanced across vcl workers? 



At the other hand, we’d like to integrate with OpenSSL 3.0.0 alpha release even without changing code in VPP master branch, so this is definitely different from sanity VPP.

 

Okay, that’s a different type of problem, unfortunately, and that does indeed need different binaries. 

 

[Yulong Pei]  I think that VPP master branch may move to build with openssl 3.0.0 sooner or later, Maybe currently VPP can provide an experiment branch built with openssl3.0.0.

 

 

Based on above problem, we’d like to separate VSAP VPP build from native VPP build.   Does it make sense?

We will look forward to the VSAP repo based build, and before this repo comes out, we’d like to change the vpp build name to avoid conflict.

 

That’s fine with me, but ultimately the CSIT maintainers need to agree to this :-)

 

Cheers,

Florin



 

Ping

 

From: Florin Coras <fcoras.lists@...> 
Sent: Monday, June 22, 2020 10:08 PM
To: Yu, Ping <ping.yu@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; vrpolak@...; Dave Wallace <dwallacelf@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

What do you mean by that, Ping? 

 

Florin




On Jun 22, 2020, at 1:42 AM, Yu, Ping <ping.yu@...> wrote:

 

Hello, all

 

I once talked with Florin that VSAP can temporarily use un-patch version, but it seems the performance is very poor, so it will not show any performance for VSAP. We’d like to consider Vratko’s solution.

Before the vsap repository comes out, we can replace vpp packet name to be vsap_vpp to avoid conflict with vpp master branch.

 

Ping

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, June 22, 2020 10:09 AM
To: Pei, Yulong <yulong.pei@...>; vrpolak@...; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

I agree with Vratko, too. 

VSAP greatly improves the performance of Nginx, which may be helpful for the promotion of VPP.

 

Best Regards

Xiaolong Jiang

 

From: Pei, Yulong <yulong.pei@...> 
Sent: Thursday, June 18, 2020 2:06 PM
To: vrpolak@...; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

Hi,

 

I agree  Vratko’s stream abstraction solution,  if no other objection,

 

1. who can help to set up  https://packagecloud.io/fdio/vsap  repository ? 

 

2. how to get permission  to manage to run  ` package_cloud push fdio/vsap  /path/to/local/vsap/package` ?

 

Best Regards

Yulong Pei

 

From: csit-dev@... <csit-dev@...> On Behalf Of Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Friday, June 12, 2020 6:42 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

I assume readers are familiar with most of the context

discussed in other communication channels.

If not, I recommend to take a peek at the long discussion

on "fdio-infra" channel of fdio slack,

starting from June 1st.

 

After some thinking, I have realized ci-management

is already conceptually prepared to handle

multiple "areas" of compatibility,

where artifacts within the same area are compatible,

but artifacts from different areas are not compatible.

 

For snapshot artifacts, the compatibility is assumed

only for the latest uploaded versions.

Temporary breakages due to API changes are tolerated.

Because of that, the concept is not named area,

but it is named "stream".

 

It is an abstraction, evolved from the concept of branch name;

needed for integration of sub-projects whose branch names do not match exactly.

(In fd.io ci-management, packagecloud repository name

matches the stream exactly, but it is not a requirement.

For example, when we used Nexus, repository name did not match directly.)

 

Anyway, in this sense, master package repository [0]

is for VPP artifacts built from VPP master branch,

and any other artifacts compatible with (latest version of) them.

VPP code in stable/2005 branch creates snapshot artifacts

uploaded to 2005 package repository [1].

 

If VSAP needs VPP builds different from both master and 2005,

it means new stream needs to be created, for example called "vsap".

VSAP merge job for vsap stream can use VSAP master branch,

and there will be no VSAP merge job for master stream.

 

In future, if VPP project accepts all VSAP-related settings,

(and VPP verify job ensures production APIs are never changed)

the following process can be applied.

A new VSAP branch (called "vsap") is cut from master branch,

and the VSAP merge job for stream vsap starts using that branch.

In VSAP master branch, the build scripts are changed

to no longer build VPP artifacts (maybe download but delete before pushing).

Then a VSAP merge job for stream master is created,

using VSAP master branch, and uploading VSAP (but not VPP) artifacts to [0].

For a period of time, both streams will be tested,

to confirm that VSAP artifacts from stream master

work equally well than VSAP (and VPP) artifacts from stream vsap.

Then, vsap branch can stop being updated and all vsap stream jobs can be deleted.

 

In other words, the "vsap" stream acts as a "feature branch" for VPP project,

except that it is maintained by VSAP (not VPP) developers.

This is more flexible than my previous idea of giving

each sub-project their own packagecloud repository.

Firstly, work can be done on multiple features.

For example streams "vsap_speed" and "vsap_scale" can at first

hardcode a particular "speed versus memory" tradeoffs,

each can be upstreamed to VPP at a different time.

Secondly, when we have more cooperating sub-projects,

they can push their artifacts into the same repository,

so that users (and testers) have easier time installing them.

 

There was some discussion on TSC meeting,

mostly related to hICN sub-project instead.

As usual, I am trying to find a solution suitable for both hICN and VSAP.

For hICN, the stream solution is to mark their merge job

(which uses hICN master branch) as stream 2005.

That assumes 2005 VPP snapshots are stable enough for them.

 

What do you think?

Any important question that does not fit into the stream abstraction?

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Tuesday, 2020-June-09 11:20
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>; Kinsella, Ray <ray.kinsella@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io
Importance: High

 

Hi Guys,

 

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

 

Best Regards

Yulong Pei

 

From: Pei, Yulong 
Sent: Monday, June 8, 2020 11:31 AM
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <XiaolongX.Jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

Hi Dave,

 

Thanks for your comments,

 

My answers inline,

 

 

 

From: Dave Wallace <dwallacelf@...> 
Sent: Wednesday, June 3, 2020 9:25 PM
To: Pei, Yulong <yulong.pei@...>; Yu, Ping <ping.yu@...>; Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [ci-management-dev] [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

Hi Yulong,

On 6/3/2020 1:00 AM, Pei, Yulong wrote:

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

 

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages.  
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.


Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test, 

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.


Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date. 

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

 

Best Regards

Yulong Pei


Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...> 
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch)<ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first. 

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping 

 

From: Florin Coras <fcoras.lists@...> 
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

 

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

 

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

 

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a

 

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

 

VSAP maybe like this:

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package topackagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package topackagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei