[csit-dev] Summary of Jenkins issues


Vanessa Valderrama
 

Andrew,

I've created a new ticket to track the merge failures. I don't believe they are related to the ongoing issues.

https://jira.linuxfoundation.org/browse/RELENG-3343

Thank you,
Vanessa

On 12/16/20 3:19 PM, Andrew 👽 Yourtchenko wrote:

Vanessa,

Today is the VPP 21.01 RC1 milestone, and two of the 21.01-RC1 merge jobs failed during the remerge, namely:



Immediate “remerge” on https://gerrit.fd.io/r/c/vpp/+/30441 didn’t take the effect until the subsequent Jenkins restart.

The RC1 milestone thus got delayed due to need to do another remerge.

Which of the three buckets that you mentioned does this occurrence fall into ?

Thanks !

--a

On 16 Dec 2020, at 21:25, Vanessa Valderrama <vvalderrama@...> wrote:



There has been some confusion regarding the current state of Jenkins and issues we're tracking. Below is a summary with the tickets where you can find and provide updates.

As per our standard support process all communication regarding these issues will be handled via the tickets.

Jenkins slowness

ci-management merge job intermittent timeouts

  • https://jira.linuxfoundation.org/browse/RELENG-3324
    • This issue is not resolved. The last steps taken were to upgrade the JJB version and increase the HTTP connectors from 100/20 to 300/50.
    • Next steps:
      • We would like to focus on the Jenkins slowness issue for now and then continue to troubleshoot the merge job timeout issue once Jenkins is stable

Jenkins build history

  • https://jira.linuxfoundation.org/browse/RELENG-3335
    • The current build discarder macro is not working properly.  We need to clean up build history and migrate to the global-jjb build discarder macro.
    • Cleaning up the existing build history will cause slowness on Jenkins. Our options are to take down time to clean up the build history and migrate jobs to the global-jjb build discarder or clean up the build history when we migrate Jenkins to the new Vexxhost V3 instance. The V3 instances are considerably cheaper and faster than the current V2 instance.
    • We would prefer to clean up the build history during the migration. We do not believe the build history is contributing to the Jenkins slowness or merge job timeout issue.