Gerrit restart is complete. Jenkins has been taken out of shutdown mode.

What:

LF is setting up a local Gerrit mirror to resolve the Gerrit cloning timeout errors that have been causing intermittent job failures. We'll need to restart Gerrit for the replication settings to take affect.

Impact:

• Jenkins will be placed in shutdown mode to allow verify and merge jobs to complete before the Gerrit restart
• Jenkins will NOT be restarted and NO jobs will need to be terminated
• Gerrit will be unavailable during the restart approximately 1 minute

---------------------------------------------------------------------------------------------------------------------------------------------

Status Update

Issue: Gateway Timeout Errors

• Summary: Intermittent Gateway Timeout Errors on the ci-management-jjb-merge jobs are causing stability issues with Jenkins causing unplanned downtime
• We have put in a change to take Nginx out of the picture and allow the build node to talk directly to Jenkins
• We'll be monitoring closely to ensure this resolves the issue

Issue: Gerrit cloning timeouts

• Summary: Intermittent job failures caused by a timeout when closing a Gerrit repo
• We have opened a Vexxhost ticket for Vexxhost and Ed Kern to troubleshoot the latency within the network the Nomad cluster is on
• We are also setting up a local Gerrit mirror which should help resolve/improve cloning - this should be complete by the end of the week
  

Issue: CSIT: s3-t21-sut1 (10.30.51.44) failure

• Summary: The device s3-t21-sut1 device is having an SSH disk read only issue and is unreachable over NW
  
• We've opened a Vexxhost ticket to check the machine

Issue: Hung jobs

• Summary: Intermittent jobs stuck/hung requiring the job to be aborted
  
• We believe this issue was resolved with the latest Jenkins upgrade
  

Please let me know if you need additional information. If you experience any hung jobs or gateway timeout errors, please open a ticket at support.linuxfoundation.org.