Topics

Jenkins Reboot Required


Vanessa Valderrama
 

Issue:
FD.io infrastructure outage

Summary:
Our vendor identified a network bug in the bridging system. The vendor performed maintenance to move OpenStack networking controls into their new DC. This required the routes and network links to be rebuilt. The changes didn't propagate properly to all of the FD.io systems.

Solution:
Restart network service

Open Action Items:
Additionally tenants were experiencing gateway timeouts connecting to the vendor auth endpoint causing new instances to fail.

Per the TSC discussion I will be killing the following Jenkins jobs, restarting Jenkins and will manually restart the jobs once Jenkins is up.

vpp-merge-master-centos7
hc2vpp-integration-master-centos7
vpp-verify-master-ubuntu1404
vpp-merge-master-ubuntu1404
hc2vpp-integration-master-ubuntu1404
hc2vpp-integration-master-ubuntu1604
vpp-docs-merge-master

Thank you,
Vanessa


Vanessa Valderrama
 

Jenkins issues are resolved, the jobs below have been restarted.

Thank you,

Vanessa

On 12/01/2016 10:42 AM, Vanessa Valderrama wrote:
Issue:
FD.io infrastructure outage

Summary:
Our vendor identified a network bug in the bridging system. The vendor performed maintenance to move OpenStack networking controls into their new DC. This required the routes and network links to be rebuilt. The changes didn't propagate properly to all of the FD.io systems.

Solution:
Restart network service

Open Action Items:
Additionally tenants were experiencing gateway timeouts connecting to the vendor auth endpoint causing new instances to fail.

Per the TSC discussion I will be killing the following Jenkins jobs, restarting Jenkins and will manually restart the jobs once Jenkins is up.

vpp-merge-master-centos7
hc2vpp-integration-master-centos7
vpp-verify-master-ubuntu1404
vpp-merge-master-ubuntu1404
hc2vpp-integration-master-ubuntu1404
hc2vpp-integration-master-ubuntu1604
vpp-docs-merge-master

Thank you,
Vanessa