Dear tsc members,

(with hicn/cicn PTL's hat on)

The current release repo https://packagecloud.io/fdio/release contains 20214 

artifacts almost entirely from our projects.

The current binary distribution is composed of 60 deb packages and 30 rpm packages.

Among hicn dependencies we have the latest VPP release, currently 20.01. 

The hICN project is composed of 6 committers and we do not support multiple releases 

but just the master HEAD. 

Our git repo has one single master branch (w/o release branches) and releases are based 

on git tags. The branch is tagged with the latest VPP release code, i.e. v20.01 which

is pushed after the VPP project has released the new version. We start catching up

with all VPP updates during the weeks of release preparation made by Andrew.

As soon as Andrew releases the new distribution under "release/" we merge our patches and 

change git tag in the hicn repo.

1 - We do not publish hicn binaries under master/ as the vpp dependencies there would not be

compatible with "hicn master" which depends on "vpp stable/"

2 - If we publish under #stable/, we'd create as many artifacts there as currently under 

"release/" and at each release we need to update ci-management. 

3 - we have no need to keep all these artifacts. So we could just keep the latest 

artifacts for each hicn release for archival reasons only. We do not backport anything 

in previous releases nor we support our user base with older releases. Packagecloud has a 

REST API to manage that well.

4 - as an alternative we could create an independent repo for the hicn project. Still we'd 

like to delete obsolete artifacts ad in option 3.

*Option 1* is unfeasible because it requires our user base to have a complex configuration 

of apt repos, which BTW only works if packages are well created. This is not always the case.

*Option 2* does not look like a solution to reduce the number of artifacts in a release folder.

It may work in conjunction with option 3. It may work well if we also get one of our committers 

in ci-management, e.g. Mauro Sardara who's substantially contributed to ci-management already.

*Option 3* seems useful in general and would allow to 

 (i) keep host configuration simple, 

 (ii) keep repo size to the right size, 

 (iii) avoid  repo duplication as in 4. 

if we keep artifacts under release our user-base would be happier as it need not host upgrades.

*Option 4 + 3* would be ok as well.

Thanks for you feedback

Best

Luca

On Thu, Jan 30, 2020 at 7:37 PM Andrew 👽 Yourtchenko <ayourtch@...> wrote:

Dear TSC,

In the capacity of the VPP 20.01 release manager, I would like to raise your attention to the fact that the fdio/release now contains 19900 files, and the there are some invalid packages, like “honeycomb”, and “-dev”.

This was found as we started CSIT testing for VPP 20.01, and Peter Mikus has a workaround - so it doesn’t impact the schedule for the 20.01 testing. (VPP packages download fine)

However, I wanted to raise this concern.

Are some projects doing so many releases, that it results in almost 20000 files?

Thanks for consideration.

--a

Luca,

 

Thank you for the excellent write-up on our meeting!

 

@Ed Warnicke, can you please create the hicn repo on packagecloud.io (https://packagecloud/fdio/hicn)?

 

-daw-

 

From: Luca Muscariello <muscariello@...>
Sent: Thursday, June 18, 2020 2:43 AM
To: Andrew 👽 Yourtchenko <ayourtch@...>; Vanessa Valderrama <vvalderrama@...>; Dave Wallace (dwallace) <dwallace@...>; tsc@...
Cc: hicn-dev@...; cicn-dev@...; ci-management-dev@...
Subject: Re: [tsc] fdio/release repository on packagecloud now contains 19900 with some odd artifacts

 

Hi,

 

After meeting with Andrew, Dave and Vanessa, we have decided to

opt for option 4 described in this thread. Tradeoffs have been 

explained in this thread.

 

This decision requires 

- the creation of a new repo https://packagecloud/fdio/hicn;

- update to ci-management on hicn JJB to push artifacts there;

- therefore previous releases are pushed manually to 

   https://packagecloud/fdio/release  via LF tickets.

Plus:

- clean up the release repo to keep actual releases only (there is

  a ci-management patch written by Mauro for this);

- lockdown releases from JJB push;

- https://packagecloud/fdio/attic is used by ci-management to cleanup

  release/ but it may be used to cleanup fdio/hicn at stationary

  regime in the long term.

 

Action points:

- merge the ci-management cleanup job

- create new packagecloud repo

- update hicn ci-management

- lockdown release

 

Thanks

Luca

 

 

 

 

 

 

 

 

On Fri, Jan 31, 2020 at 9:51 AM Luca Muscariello <muscariello@...> wrote:

Dear tsc members,

 

(with hicn/cicn PTL's hat on)

 

The current release repo https://packagecloud.io/fdio/release contains 20214 

artifacts almost entirely from our projects.

The current binary distribution is composed of 60 deb packages and 30 rpm packages.

 

Among hicn dependencies we have the latest VPP release, currently 20.01. 

The hICN project is composed of 6 committers and we do not support multiple releases 

but just the master HEAD. 

 

Our git repo has one single master branch (w/o release branches) and releases are based 

on git tags. The branch is tagged with the latest VPP release code, i.e. v20.01 which

is pushed after the VPP project has released the new version. We start catching up

with all VPP updates during the weeks of release preparation made by Andrew.

As soon as Andrew releases the new distribution under "release/" we merge our patches and 

change git tag in the hicn repo.

 

1 - We do not publish hicn binaries under master/ as the vpp dependencies there would not be

compatible with "hicn master" which depends on "vpp stable/"

 

2 - If we publish under #stable/, we'd create as many artifacts there as currently under 

"release/" and at each release we need to update ci-management. 

 

3 - we have no need to keep all these artifacts. So we could just keep the latest 

artifacts for each hicn release for archival reasons only. We do not backport anything 

in previous releases nor we support our user base with older releases. Packagecloud has a 

REST API to manage that well.

 

4 - as an alternative we could create an independent repo for the hicn project. Still we'd 

like to delete obsolete artifacts ad in option 3.

 

*Option 1* is unfeasible because it requires our user base to have a complex configuration 

of apt repos, which BTW only works if packages are well created. This is not always the case.

 

*Option 2* does not look like a solution to reduce the number of artifacts in a release folder.

It may work in conjunction with option 3. It may work well if we also get one of our committers 

in ci-management, e.g. Mauro Sardara who's substantially contributed to ci-management already.

 

*Option 3* seems useful in general and would allow to 

 (i) keep host configuration simple, 

 (ii) keep repo size to the right size, 

 (iii) avoid  repo duplication as in 4. 

if we keep artifacts under release our user-base would be happier as it need not host upgrades.

 

*Option 4 + 3* would be ok as well.

 

Thanks for you feedback

Best

Luca

 

 

 

On Thu, Jan 30, 2020 at 7:37 PM Andrew 👽 Yourtchenko <ayourtch@...> wrote:

Dear TSC,

In the capacity of the VPP 20.01 release manager, I would like to raise your attention to the fact that the fdio/release now contains 19900 files, and the there are some invalid packages, like “honeycomb”, and “-dev”.

This was found as we started CSIT testing for VPP 20.01, and Peter Mikus has a workaround - so it doesn’t impact the schedule for the 20.01 testing. (VPP packages download fine)

However, I wanted to raise this concern.

Are some projects doing so many releases, that it results in almost 20000 files?

Thanks for consideration.

--a