Before we enable VSAP merge job,

I would like to understand the build process more.

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

Observations (and some comments) first:

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

The job was triggered for VSAP master head. Ok.

 > git config --get submodule.vpp.url # timeout=10

Vpp repository is acts as a submodule for VSAP. Ok.

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

Openssl is built, I assume there is a good reason to do that.

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

I believe this is the main output the VSAP merge job will upload to packagecloud.

Now the main question.

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

Vratko.

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

Forgot to add Ping & Xiaolong in the email To list.

Hello Dave,

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

Best Regards

Yulong Pei

> Just like VPP:

>> https://packagecloud.io/fdio/master

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

>> https://packagecloud.io/fdio/vsap/master

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

@Dave Wallace

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

Vratko.

[2] https://packagecloud.io/fdio

[3] https://github.com/FDio/ci-management/blob/be612c3280a8e8e7a67431bf2f8aa55c81b255cb/jjb/scripts/packagecloud_push.sh#L2

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

The latter.

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

>https://packagecloud.io/fdio/master

VSAP maybe like this:

>https://packagecloud.io/fdio/vsap/master

Before we enable VSAP merge job,

I would like to understand the build process more.

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

Observations (and some comments) first:

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

The job was triggered for VSAP master head. Ok.

 > git config --get submodule.vpp.url # timeout=10

Vpp repository is acts as a submodule for VSAP. Ok.

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

Openssl is built, I assume there is a good reason to do that.

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

I believe this is the main output the VSAP merge job will upload to packagecloud.

Now the main question.

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

Vratko.

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

Forgot to add Ping & Xiaolong in the email To list.

Hello Dave,

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

Best Regards

Yulong Pei

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Best Regards

Yulong Pei

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

Ping

Hi Ping, 

Hi Dave,

Thanks for your comments,

My answers inline,

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.

Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.

Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

Best Regards

Yulong Pei

Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

[1] https://gerrit.fd.io/r/c/vpp/+/25663

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

>> https://packagecloud.io/fdio/master

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

>> https://packagecloud.io/fdio/vsap/master

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

@Dave Wallace

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

[2] https://packagecloud.io/fdio

[3] https://github.com/FDio/ci-management/blob/be612c3280a8e8e7a67431bf2f8aa55c81b255cb/jjb/scripts/packagecloud_push.sh#L2

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

>https://packagecloud.io/fdio/master

 

VSAP maybe like this:

>https://packagecloud.io/fdio/vsap/master

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 

Hi Guys,

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

Best Regards

Yulong Pei

Hi Dave,

Thanks for your comments,

My answers inline,

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.

Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.

Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

Best Regards

Yulong Pei

Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

[1] https://gerrit.fd.io/r/c/vpp/+/25663

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

>> https://packagecloud.io/fdio/master

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

>> https://packagecloud.io/fdio/vsap/master

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

@Dave Wallace

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

[2] https://packagecloud.io/fdio

[3] https://github.com/FDio/ci-management/blob/be612c3280a8e8e7a67431bf2f8aa55c81b255cb/jjb/scripts/packagecloud_push.sh#L2

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

>https://packagecloud.io/fdio/master

 

VSAP maybe like this:

>https://packagecloud.io/fdio/vsap/master

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 

 

I assume readers are familiar with most of the context

discussed in other communication channels.

If not, I recommend to take a peek at the long discussion

on "fdio-infra" channel of fdio slack,

starting from June 1st.

After some thinking, I have realized ci-management

is already conceptually prepared to handle

multiple "areas" of compatibility,

where artifacts within the same area are compatible,

but artifacts from different areas are not compatible.

For snapshot artifacts, the compatibility is assumed

only for the latest uploaded versions.

Temporary breakages due to API changes are tolerated.

Because of that, the concept is not named area,

but it is named "stream".

It is an abstraction, evolved from the concept of branch name;

needed for integration of sub-projects whose branch names do not match exactly.

(In fd.io ci-management, packagecloud repository name

matches the stream exactly, but it is not a requirement.

For example, when we used Nexus, repository name did not match directly.)

Anyway, in this sense, master package repository [0]

is for VPP artifacts built from VPP master branch,

and any other artifacts compatible with (latest version of) them.

VPP code in stable/2005 branch creates snapshot artifacts

uploaded to 2005 package repository [1].

If VSAP needs VPP builds different from both master and 2005,

it means new stream needs to be created, for example called "vsap".

VSAP merge job for vsap stream can use VSAP master branch,

and there will be no VSAP merge job for master stream.

In future, if VPP project accepts all VSAP-related settings,

(and VPP verify job ensures production APIs are never changed)

the following process can be applied.

A new VSAP branch (called "vsap") is cut from master branch,

and the VSAP merge job for stream vsap starts using that branch.

In VSAP master branch, the build scripts are changed

to no longer build VPP artifacts (maybe download but delete before pushing).

Then a VSAP merge job for stream master is created,

using VSAP master branch, and uploading VSAP (but not VPP) artifacts to [0].

For a period of time, both streams will be tested,

to confirm that VSAP artifacts from stream master

work equally well than VSAP (and VPP) artifacts from stream vsap.

Then, vsap branch can stop being updated and all vsap stream jobs can be deleted.

In other words, the "vsap" stream acts as a "feature branch" for VPP project,

except that it is maintained by VSAP (not VPP) developers.

This is more flexible than my previous idea of giving

each sub-project their own packagecloud repository.

Firstly, work can be done on multiple features.

For example streams "vsap_speed" and "vsap_scale" can at first

hardcode a particular "speed versus memory" tradeoffs,

each can be upstreamed to VPP at a different time.

Secondly, when we have more cooperating sub-projects,

they can push their artifacts into the same repository,

so that users (and testers) have easier time installing them.

There was some discussion on TSC meeting,

mostly related to hICN sub-project instead.

As usual, I am trying to find a solution suitable for both hICN and VSAP.

For hICN, the stream solution is to mark their merge job

(which uses hICN master branch) as stream 2005.

That assumes 2005 VPP snapshots are stable enough for them.

What do you think?

Any important question that does not fit into the stream abstraction?

Vratko.

[0] https://packagecloud.io/fdio/master

[1] https://packagecloud.io/fdio/2005

Hi Guys,

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

Best Regards

Yulong Pei

Hi Dave,

Thanks for your comments,

My answers inline,

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.

Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.

Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

Best Regards

Yulong Pei

Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

[1] https://gerrit.fd.io/r/c/vpp/+/25663

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

 

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

 

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

 

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

>> https://packagecloud.io/fdio/master

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

>> https://packagecloud.io/fdio/vsap/master

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

@Dave Wallace

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

[2] https://packagecloud.io/fdio

[3] https://github.com/FDio/ci-management/blob/be612c3280a8e8e7a67431bf2f8aa55c81b255cb/jjb/scripts/packagecloud_push.sh#L2

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

>https://packagecloud.io/fdio/master

 

VSAP maybe like this:

>https://packagecloud.io/fdio/vsap/master

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei

 

 

 

Hi,

I agree  Vratko’s stream abstraction solution,  if no other objection,

1. who can help to set up  https://packagecloud.io/fdio/vsap  repository ?

2. how to get permission  to manage to run  ` package_cloud push fdio/vsap  /path/to/local/vsap/package` ?

Best Regards

Yulong Pei

I assume readers are familiar with most of the context

discussed in other communication channels.

If not, I recommend to take a peek at the long discussion

on "fdio-infra" channel of fdio slack,

starting from June 1st.

After some thinking, I have realized ci-management

is already conceptually prepared to handle

multiple "areas" of compatibility,

where artifacts within the same area are compatible,

but artifacts from different areas are not compatible.

For snapshot artifacts, the compatibility is assumed

only for the latest uploaded versions.

Temporary breakages due to API changes are tolerated.

Because of that, the concept is not named area,

but it is named "stream".

It is an abstraction, evolved from the concept of branch name;

needed for integration of sub-projects whose branch names do not match exactly.

(In fd.io ci-management, packagecloud repository name

matches the stream exactly, but it is not a requirement.

For example, when we used Nexus, repository name did not match directly.)

Anyway, in this sense, master package repository [0]

is for VPP artifacts built from VPP master branch,

and any other artifacts compatible with (latest version of) them.

VPP code in stable/2005 branch creates snapshot artifacts

uploaded to 2005 package repository [1].

If VSAP needs VPP builds different from both master and 2005,

it means new stream needs to be created, for example called "vsap".

VSAP merge job for vsap stream can use VSAP master branch,

and there will be no VSAP merge job for master stream.

In future, if VPP project accepts all VSAP-related settings,

(and VPP verify job ensures production APIs are never changed)

the following process can be applied.

A new VSAP branch (called "vsap") is cut from master branch,

and the VSAP merge job for stream vsap starts using that branch.

In VSAP master branch, the build scripts are changed

to no longer build VPP artifacts (maybe download but delete before pushing).

Then a VSAP merge job for stream master is created,

using VSAP master branch, and uploading VSAP (but not VPP) artifacts to [0].

For a period of time, both streams will be tested,

to confirm that VSAP artifacts from stream master

work equally well than VSAP (and VPP) artifacts from stream vsap.

Then, vsap branch can stop being updated and all vsap stream jobs can be deleted.

In other words, the "vsap" stream acts as a "feature branch" for VPP project,

except that it is maintained by VSAP (not VPP) developers.

This is more flexible than my previous idea of giving

each sub-project their own packagecloud repository.

Firstly, work can be done on multiple features.

For example streams "vsap_speed" and "vsap_scale" can at first

hardcode a particular "speed versus memory" tradeoffs,

each can be upstreamed to VPP at a different time.

Secondly, when we have more cooperating sub-projects,

they can push their artifacts into the same repository,

so that users (and testers) have easier time installing them.

There was some discussion on TSC meeting,

mostly related to hICN sub-project instead.

As usual, I am trying to find a solution suitable for both hICN and VSAP.

For hICN, the stream solution is to mark their merge job

(which uses hICN master branch) as stream 2005.

That assumes 2005 VPP snapshots are stable enough for them.

What do you think?

Any important question that does not fit into the stream abstraction?

Vratko.

[0] https://packagecloud.io/fdio/master

[1] https://packagecloud.io/fdio/2005

Hi Guys,

If a packagecloud sandbox is not a good way,  could you kindly help to give suggestion about how to proceed this work ?  Sorry to mark this email with high importance since it is really blocked for a long time.

Best Regards

Yulong Pei

Hi Dave,

Thanks for your comments,

My answers inline,

Anyway,  could we have a temporary repo for VSAP at packagecloud?  then Jenkins build and verify job for VSAP work and CSIT test cases for TLS with VSAP work can be proceeded,  they are blocked for a long time 😊,

This is a possibility as long as there are some controls in place to limit access to the repo to CI jobs and/or periodic purging of the repo artifacts.  However, this does come with the cost of overseeing/managing the repo contents so is not the optimal solution IMHO.

[Yulong Pei]  could we regard “temporary repo” as a packagecloude sandbox, just like fdio Jenkins sandbox ? It do not need keep it for a long time.

When issue about VSAP specific patch merged to VPP master was fixed,   then we can move VSAP built package to standard repo at packagecloud at that time.

Once code is merged to VPP then it becomes available in the regular VPP packages.  VSAP built packages are temporary thus will never be moved.

However, this model introduces an additional layer of CI jobs to test those features that exist in VPP plus those features that are only supported in the VSAP project packages. 
There are limitations on the availability of CSIT testbeds to cover VPP features.  Therefore duplication test execution against the VSAP artifacts and VPP artifacts is not feasible without additional testbed resources dedicated to the VSAP project.  So this will require movement of test cases from VSAP to VPP which may introduce additional challenges. 

[Yulong Pei]  Currently we can limit VSAP test only run on one skx or clx CSIT testbed, trigger one test weekly is enough,  and I think it will deploy more ICELake testbeds in CSIT lab in the future.

Another issue is ongoing trending & failure analysis of the CSIT test results of the VSAP artifacts.  The CSIT & VPP teams work together on identifying & resolving test failures and performance regressions.  Managing all of that for VPP and VSAP is going to be very challenging.  IMHO, it is unrealistic to expect the CSIT & VPP teams to handle VSAP in addition to VPP.  How much additional work this is for CSIT & VSAP is not currently understood.

[Yulong Pei]  Actually  VSAP test  is to show  VPP TLS/tcp performance,  VSAP is only a modified Nginx, it change Nginx socket layer to fit for VPP VCL or LDP,  we may also regard VSAP(Nginx) as a test tool  for VPP TLS/tcp test,

As you know, VPP has a internal http_static server, but if test VPP TLS/tcp with that,  the performance is  bad.  so from test view, VSAP test is VPP test,  not Nginx test.  And Intel team also can help on failure analysis.

Given the number of unknowns in the process and deployment/consumption of resources (both human & automation), I think that it would be best to pick a very small set of tests (maybe as small as 1) as an initial deployment of the project CI pipeline.  Then evaluate/refactor the pipeline before including everything that you have implemented to date.

[Yulong Pei]   I agree that it is better to pick small set of tests as an initial deployment of the project CI pipeline.

Best Regards

Yulong Pei

Thanks,
-daw-

 

Best Regards

Yulong Pei

 

From: Yu, Ping <ping.yu@...>
Sent: Wednesday, June 3, 2020 11:04 AM
To: Florin Coras <fcoras.lists@...>; Andrew Yourtchenko (ayourtch) <ayourtch@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>; Yu, Ping <ping.yu@...>; Yu, Ping <ping.yu@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Yes, for this specific issue, we’d like to move forward, and as a general packaging, there is definitely some code which requires customize VPP. For example, some company wants us to give them the version for CBDMA, and we also consider to release this code to VSAP first.

 

@Andrew Yourtchenko (ayourtch)fixme means there is some problem, but actually it is just not merged to master branch yet. This is quite different hint to users.

 

Ping

 

From: Florin Coras <fcoras.lists@...>
Sent: Wednesday, June 3, 2020 6:57 AM
To: Yu, Ping <ping.yu@...>
Cc: Andrew Yourtchenko (ayourtch) <ayourtch@...>; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping, 

 

I guess there are two improvements that have not entirely made their way into vpp:

- Pinning vpp workers to vcl workers. If we have a patch that’s generic enough, we could try merging it into master. 

- A non-locking vls alternative. I believe [1] considerably improved locking performance for multi-process applications. Do we know if vls is still a bottleneck after this? 

 

To get things moving, could we start by using existing vpp packages in combination with vsap nginx? That would give us time to clarify the two points above and a performance baseline. 

 

Regards,

Florin

 

[1] https://gerrit.fd.io/r/c/vpp/+/25663

 

 

On Jun 2, 2020, at 3:43 AM, Yu, Ping <ping.yu@...> wrote:

 

Okay, the idea to build a different name is okay, but we’d like to call it as “VSAP_vpp”.

 

For the reason why some patches could not merged into VPP master branch is that we are now working to make VPP host stack work smoothly in Nginx, and Nginx is a single-thread multiple process application, thus we can remove some “lock” mechanism in VPP to speed up the performance. We also make a group-partition method to speed up the performance in Nginx, but community (florin) think it is not a general solution yet, and we’d like to enhance it, but some customers need this patch to get their Nginx speed up.  

 

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Tuesday, June 2, 2020 5:18 PM
To: Yu, Ping <ping.yu@...>
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hi Ping,

 

On 2 Jun 2020, at 11:05, Yu, Ping <ping.yu@...> wrote:

 

Hi, Andrew

 

You know, VPP has made some changes for DPDK, and not all of the patches are merged into DPDK master branch. It is the same in VSAP.

 

The difference is VPP doesn’t try to upload the patched version of DPDK, does it ?

 

The VPP patch in VSAP is definitely useful, but it is not necessary in VPP master branch due to complicated considerations.

 

Could you try to outline the considerations or give a pointer where I can read up?

 

I also don’t think it is completely reasonable that someone install all fd.io project at once. If someone wants to use VSAP,   some customized VPP is required, just VPP wants to customize DPDK.  

 

Do you add a custom repository for each package that you install on your system ?

 

The solution is not difficult, and just provide repo for each project.

 

Private repos are a workaround to not having a proper packaging. I maintain that having more than one package called “vpp” in the FD.io repo that does slightly different thing is not the right thing to do. 

 

This needs to be discussed at TSC I think. 

 

Till then I would suggest for VSAP to add another patch to VPP which patches the packaging to rename all the “vpp/VPP” references in the packages into “fixmevsap/FIXMEVSAP”. 

 

This way there is no confusion, no need for separate repos and no packaging conflict, the packaging issue is clearly highlighted, and you are not blocked in the interim before the TSC?

 

—a

 

 

Ping

 

From: Andrew Yourtchenko (ayourtch) <ayourtch@...> 
Sent: Monday, June 1, 2020 10:26 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>
Cc: Jiang, XiaolongX <xiaolongx.jiang@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

 

1) it is not completely unreasonable to imagine someone want to install all of the FD.io projects at once, so the process and workflow must allow for that. Debian has thousands of packages maintained by thousands of volunteers in the same repo. 

 

2) having the artifacts with the same name and description but different code is a road to a lot of pain for everyone. If a patch is useful for VPP consumption, it must be in VPP master, and there should be only one version of the master artifacts, then the VSAP should be able to act as a plugin and/or consume VPP as a wholesale dependency.

 

What are the obstacles that prevent that ?

 

--a

 

On 1 Jun 2020, at 15:39, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> wrote:

 

> Just like VPP:

>> https://packagecloud.io/fdio/master

 

Currently, it is a "fdio" repository,

projects other than VPP are uploading there too.

 

> In order to avoid confusion, we would like to have a new packagecloud repository.

> VSAP maybe like this:

>> https://packagecloud.io/fdio/vsap/master

 

So this is a different pattern,

but I agree it is probably the simplest solution

for allowing different project to upload packages of the same name.

Later you will probably also need /vsap/release repo.

 

Looking at [2], there already are several repositories

unrelated to STREAM variable.

So creating vsap/master could work,

unless packagecloud limits the repository level.

In that case we could create something like vsap-master.

 

The main issue will be in the merge job,

as packagecloud_push.sh directly uses PCIO_CO [3] env var.

You would need to introduce a new environment variable

(to pass "vsap" while keeping STREAM for "master")

and change the logic to insert that everywhere between ${PCIO_CO} and ${STREAM}

(without affecting jobs that do not set the new environment variable).

 

@Dave Wallace

What do you think?

Should we start creating per-project repositories,

or should we keep trying to reuse fdio/master?

I personally prefer per-project repositories,

just so I do not have to investigate console output

of every new project verify/merge job. :)

 

I think it would also help with HICN release frequency,

if users could install VPP from fdio/2005 and HICN from fdio/hicn/master.

 

Vratko.

 

[2] https://packagecloud.io/fdio

[3] https://github.com/FDio/ci-management/blob/be612c3280a8e8e7a67431bf2f8aa55c81b255cb/jjb/scripts/packagecloud_push.sh#L2

 

From: Jiang, XiaolongX <xiaolongx.jiang@...> 
Sent: Monday, 2020-June-01 09:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>; Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

> When installing the nginx .deb package,

> does it contain statically linked parts of VPP it needs,

> or does a user need to also install the patched vpp .deb packages?

 

The latter.

 

 

> If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

 

In order to avoid confusion, we would like to have a new packagecloud repository.

Just like VPP:

>https://packagecloud.io/fdio/master

 

VSAP maybe like this:

>https://packagecloud.io/fdio/vsap/master

 

 

Best Regards

Xiaolong Jiang

 

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...> 
Sent: Friday, May 29, 2020 10:08 PM
To: Pei, Yulong <yulong.pei@...>; Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: RE: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Before we enable VSAP merge job,

I would like to understand the build process more.

 

So I looked at verify job console output.

(Link [0] will stop working when Sandbox is cleared over weekend.)

 

Observations (and some comments) first:

 

 > git fetch --tags --progress -- git://10.30.48.3/mirror/vsap refs/heads/master # timeout=10

 

The job was triggered for VSAP master head. Ok.

 

 > git config --get submodule.vpp.url # timeout=10

 

Vpp repository is acts as a submodule for VSAP. Ok.

 

--- building openssl 3.0.0-alpha2 - log: /w/workspace/vsap-verify-master-ubuntu1804-vcl/_build/openssl.build.log

 

Openssl is built, I assume there is a good reason to do that.

 

Applying patch: 0001-3.0.0.patch

patching file src/plugins/crypto_openssl/CMakeLists.txt

 

So VSAP does not use "vanilla" VPP,

but applies some patches.

That is normal during the development,

VSAP working with the patched VPP

acts as a kind of indirect verify job

for upstreaming the patch into VPP repo.

 

dpkg-deb: building package 'vpp' in '../vpp_20.09-rc0~73-ge7df8cbb6~b19_amd64.deb'.

 

Note that packagecloud already contains a vpp ~73 .deb [1],

it comes from VPP merge job.

The dangerous part is that even the ge part is the same,

despite the patches being applied here.

Only the ~b part happens to be different,

but in production also that could happen to be the same.

 

{:timestamp=>"2020-05-29T03:56:39.897262+0000", :message=>"Created package", :path=>"/w/workspace/vsap-verify-master-ubuntu1804-vcl/_install/deb-vcl/nginx_1.14.2-ubuntu18.04_amd64.deb"}

 

I believe this is the main output the VSAP merge job will upload to packagecloud.

 

Now the main question.

 

When installing the nginx .deb package,

does it contain statically linked parts of VPP it needs,

or does a user need to also install the patched vpp .deb packages?

 

If the former, the VSAP build process should make sure

any unneeded .deb files are deleted (or better yet not even created)

before proceeding to packagecloud upload.

 

If the latter, how do we distinguish "vanilla" VPP from "vsap" VPP?

Uploading them to the same packagecloud repository will certainly confuse users.

One possibility is to prepend vsap- prefix to all VPP packages created by VSAP build.

Short term by patching VPP makefiles,

longer-term by upstreaming a Change that takes prefix from an environment variable.

 

Vratko.

 

[0] https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl/19/consoleText

[1] https://packagecloud.io/fdio/master/packages/ubuntu/bionic/vpp_20.09-rc0~73-ge7df8cbb6~b1324_amd64.deb

 

From: csit-dev@... <csit-dev@...> On Behalf Of Pei, Yulong
Sent: Friday, 2020-May-29 08:21
To: Dave Wallace <dwallacelf@...>; Yu, Ping <ping.yu@...>; Jiang, XiaolongX <xiaolongx.jiang@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) <vrpolak@...>;maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) <pmikus@...>
Subject: Re: [csit-dev] About how to upload package to packagecloud.io and download package from packagecloud.io

 

Forgot to add Ping & Xiaolong in the email To list.

 

From: Pei, Yulong 
Sent: Friday, May 29, 2020 2:19 PM
To: Dave Wallace <dwallacelf@...>
Cc: csit-dev@...; ci-management-dev@...; Vratko Polak <vrpolak@...>; maciek@...; Kinsella, Ray <ray.kinsella@...>; Peter Mikus <pmikus@...>
Subject: About how to upload package to packagecloud.io and download package from packagecloud.io

 

Hello Dave, 

 

Could you kindly help about how to upload package to packagecloud.io and download package from packagecloud.io ?

 

Currently we are working on to integrate FDIO VSAP project build  to FDIO infrastructure, we can add build job to ci-management, example as below,

 

https://jenkins.fd.io/sandbox/job/vsap-verify-master-ubuntu1804-vcl

 

Next step it need to upload the VSAP built packages to packagecloud.io,  I guess that it may need set up a repo for VSAP project in packageclould.io,  right ?

 

And then we also need download VSAP built packages from packagecloud.io for CSIT to run VSAP performance test,  to use which way to get this ?

 

Best Regards

Yulong Pei