Re: fdio/release repository on packagecloud now contains 19900 with some odd artifacts

Luca Muscariello

Dear Andrew,

thanks for bringing this to our attention.

As mitigation I'll file another ticket to LF to remove unnecessary packages.
Concerning the long term solution I'm not sure which one has been agreed.

We did not get much feedback on how to proceed except from you.
We had a pending vote to get Mauro approved as committer in ci-management,
as these options require us to have someone to pay attention to hicn patches
during the release phase.

Please share your opinion on what is preferable so that we can implement it.

Thanks and apologies if that is not yet fixed.

On Wed, May 6, 2020 at 1:03 PM Andrew 👽 Yourtchenko <ayourtch@...> wrote:
Dear Luca,

As we are approaching the VPP release 20.05, I wanted to revive this thread.

At the moment of this writing, the fdio/release has 7408 packages.

Could you please give an update where you are with the implementation of the plans we discussed ? (As per the quoted mail thread).

Thanks a lot !


On 7 Feb 2020, at 11:40, Luca Muscariello <muscariello@...> wrote:


Agree, this is just mitigation. 
4+3 is something we could work on in ci-management and plan to deploy for the next release.

Golden releases, as you say, is to be understood for hicn, we don't have muscles to backport bug fixes in golden releases.
We would end up putting (tomb)stone releases :-) 
For the time being we stay cheap and little and maintain one release.


On Fri, Feb 7, 2020 at 11:28 AM Andrew 👽 Yourtchenko <ayourtch@...> wrote:
Excellent, thanks Luca.

Longer term I still suggest option 4+3, and if you have “golden” releases (which in turn will work with released VPP versions) then you might put them under the fdio/release, thus replicating the behavior of VPP repos.


On 7 Feb 2020, at 11:21, Luca Muscariello <muscariello@...> wrote:


we have reduced the number of artifacts to 3200.

This should fix the problem.


On Fri, Jan 31, 2020 at 9:51 AM Luca Muscariello <muscariello@...> wrote:
Dear tsc members,

(with hicn/cicn PTL's hat on)

The current release repo contains 20214 
artifacts almost entirely from our projects.
The current binary distribution is composed of 60 deb packages and 30 rpm packages.

Among hicn dependencies we have the latest VPP release, currently 20.01. 
The hICN project is composed of 6 committers and we do not support multiple releases 
but just the master HEAD. 

Our git repo has one single master branch (w/o release branches) and releases are based 
on git tags. The branch is tagged with the latest VPP release code, i.e. v20.01 which
is pushed after the VPP project has released the new version. We start catching up
with all VPP updates during the weeks of release preparation made by Andrew.
As soon as Andrew releases the new distribution under "release/" we merge our patches and 
change git tag in the hicn repo.

1 - We do not publish hicn binaries under master/ as the vpp dependencies there would not be
compatible with "hicn master" which depends on "vpp stable/"

2 - If we publish under #stable/, we'd create as many artifacts there as currently under 
"release/" and at each release we need to update ci-management. 

3 - we have no need to keep all these artifacts. So we could just keep the latest 
artifacts for each hicn release for archival reasons only. We do not backport anything 
in previous releases nor we support our user base with older releases. Packagecloud has a 
REST API to manage that well.

4 - as an alternative we could create an independent repo for the hicn project. Still we'd 
like to delete obsolete artifacts ad in option 3.

*Option 1* is unfeasible because it requires our user base to have a complex configuration 
of apt repos, which BTW only works if packages are well created. This is not always the case.

*Option 2* does not look like a solution to reduce the number of artifacts in a release folder.
It may work in conjunction with option 3. It may work well if we also get one of our committers 
in ci-management, e.g. Mauro Sardara who's substantially contributed to ci-management already.

*Option 3* seems useful in general and would allow to 
 (i) keep host configuration simple, 
 (ii) keep repo size to the right size, 
 (iii) avoid  repo duplication as in 4. 
if we keep artifacts under release our user-base would be happier as it need not host upgrades.

*Option 4 + 3* would be ok as well.

Thanks for you feedback

On Thu, Jan 30, 2020 at 7:37 PM Andrew 👽 Yourtchenko <ayourtch@...> wrote:
Dear TSC,

In the capacity of the VPP 20.01 release manager, I would like to raise your attention to the fact that the fdio/release now contains 19900 files, and the there are some invalid packages, like “honeycomb”, and “-dev”.

This was found as we started CSIT testing for VPP 20.01, and Peter Mikus has a workaround - so it doesn’t impact the schedule for the 20.01 testing. (VPP packages download fine)

However, I wanted to raise this concern.

Are some projects doing so many releases, that it results in almost 20000 files?

Thanks for consideration.


Join to automatically receive all group messages.