|
Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES@Cisco) <mgradzki@...>
+ hc2vpp list (hc2vpp csit jobs are failing with the same error message)
Regards,
Marek
toggle quoted message
Show quoted text
From: csit-dev-bounces@... [mailto:csit-dev-bounces@...]
On Behalf Of Peter Mikus -X (pmikus - PANTHEON TECHNOLOGIES at Cisco)
Sent: 2 stycznia 2018 06:37
To: Ed Kern (ejk) <ejk@...>
Cc: csit-dev@...; vpp-dev@...
Subject: Re: [csit-dev] [vpp-dev] gerrit 9904 VIRL verification is failing
Hello Ed,
Can you please take a look? Looks like VIRLs are trying to connect to Cisco master machines but communication failing (due to company shutdown?)?
Can you please closer describe why this connection is required?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems Limited
On 12/27/2017 09:02 AM, Neale Ranns (nranns) wrote:
Hi Nitin,
Hit the ‘reply’ button and post a review comment of:
recheck
that will poke Jenkins to redo the verification.
/neale
-----Original Message-----
From: <vpp-dev-bounces@...> on behalf of "Saxena, Nitin" <Nitin.Saxena@...>
Date: Wednesday, 27 December 2017 at 14:38
To: "Dave Barach (dbarach)" <dbarach@...>, "vpp-dev@..." <vpp-dev@...>
Subject: [vpp-dev] gerrit 9904 VIRL verification is failing
Hi,
I sent a patch (https://gerrit.fd.io/r/#/c/9904/) for review in which "vpp-csit-verify-virl-master" job is failing.
Console logs (https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/8798/console.log.gz) shows following error.
====================
call_home\nFlmClientException: Cisco contact was not established. This may be temporary.\nPlease make sure the VIRL server is connected to the Internet and capable of reaching the configured Cisco master.\nAlso make sure that the minion key provided to you matches your minion ID and domain, and remains valid.\nCurrent status is: Last successful contact was more than 7 days ago.\nLast call home check result was: Call has timed out; failed to connect or minion key not accepted.\n"
I have also seen what looks like the same thing on VIRL3 which is currently not in production. I reported it yesterday to the CSIT mailing list.
https://jenkins.fd.io/job/csit-vpp-functional-master-ubuntu1604-virl/3290/
}
+ VIRL_SID[${index}]=
+ retval=1
+ '[' 1 -ne 0 ']'
+ echo 'VIRL simulation start failed on 10.30.51.29'
VIRL simulation start failed on 10.30.51.29
=======================
Seems like a temporary problem. What is the gerrit command such that Jenkins again start doing verification.
Thanks,
Nitin
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
|
|
|
peter was close…the virl salt server we were pointed at did have a hard crash..
ive repointed all 3 (this also should correct the issue therbert was seeing on virl3)
at production list of masters and verified all three as getting good license response now.
Let me know if you see issues on your rechecks and ill give it a harder reset swat.
On Jan 2, 2018, at 10:04 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) < mgradzki@...> wrote:
+ hc2vpp list (hc2vpp csit jobs are failing with the same error message)
Regards,
Marek
Hello Ed,
Can you please take a look? Looks like VIRLs are trying to connect to Cisco master machines but communication failing (due to company shutdown?)?
Can you please closer describe why this connection is required?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems Limited
On 12/27/2017 09:02 AM, Neale Ranns (nranns) wrote:
Hi Nitin,
Hit the ‘reply’ button and post a review comment of:
recheck
that will poke Jenkins to redo the verification.
/neale
-----Original Message-----
From: <vpp-dev-bounces@...> on behalf of "Saxena, Nitin" <Nitin.Saxena@...>
Date: Wednesday, 27 December 2017 at 14:38
To: "Dave Barach (dbarach)" <dbarach@...>, "vpp-dev@..." <vpp-dev@...>
Subject: [vpp-dev] gerrit 9904 VIRL verification is failing
Hi,
I sent a patch (https://gerrit.fd.io/r/#/c/9904/) for review in which "vpp-csit-verify-virl-master" job is failing.
Console logs (https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/8798/console.log.gz) shows following error.
====================
call_home\nFlmClientException: Cisco contact was not established. This may be temporary.\nPlease make sure the VIRL server is connected to the Internet and capable of reaching the configured Cisco master.\nAlso make sure that the minion key provided to you matches your minion ID and domain, and remains valid.\nCurrent status is: Last successful contact was more than 7 days ago.\nLast call home check result was: Call has timed out; failed to connect or minion key not accepted.\n"
I have also seen what looks like the same thing on VIRL3 which is currently not in production. I reported it yesterday to the CSIT mailing list.
https://jenkins.fd.io/job/csit-vpp-functional-master-ubuntu1604-virl/3290/
}
+ VIRL_SID[${index}]=
+ retval=1
+ '[' 1 -ne 0 ']'
+ echo 'VIRL simulation start failed on 10.30.51.29'
VIRL simulation start failed on 10.30.51.29
=======================
Seems like a temporary problem. What is the gerrit command such that Jenkins again start doing verification.
Thanks,
Nitin
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
|
|
|
Thomas F Herbert <therbert@...>
Ed,
Thanks for fixing this.
On 01/02/2018 12:51 PM, Ed Kern (ejk)
wrote:
peter was close…the virl salt server we were pointed at did have a
hard crash..
ive repointed all 3 (this also should correct the
issue therbert was seeing on virl3)
at production list of masters and verified all three
as getting good license response now.
Ed, if it due to a limited resource that is invoked VIRL client
starts on all 3 servers, is it possible that the load testing I was
doing on VIRL3 broke it or at least exposed a generic load testing
issue common to all VIRL servers.? I first saw it fail when I
rechecked 3 close to simultaneous rechecks on VIRL3 and a short time
after all rechecks failed. Before It failed,
Let me know if you see issues on your rechecks and
ill give it a harder reset swat.
Ed
On Jan 2, 2018, at 10:04 AM, Marek Gradzki -X
(mgradzki - PANTHEON TECHNOLOGIES at Cisco) < mgradzki@...> wrote:
+
hc2vpp list (hc2vpp csit jobs are failing with the
same error message)
Regards,
Marek
Hello Ed,
Can you please take a
look? Looks like VIRLs are trying to connect to
Cisco master machines but communication failing (due
to company shutdown?)?
Can you please closer
describe why this connection is required?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems
Limited
On 12/27/2017 09:02 AM, Neale Ranns (nranns) wrote:
Hi Nitin,
Hit the ‘reply’ button and post a review comment of:
recheck
that will poke Jenkins to redo the verification.
/neale
-----Original Message-----
From: <vpp-dev-bounces@...> on behalf of "Saxena, Nitin" <Nitin.Saxena@...>
Date: Wednesday, 27 December 2017 at 14:38
To: "Dave Barach (dbarach)" <dbarach@...>, "vpp-dev@..." <vpp-dev@...>
Subject: [vpp-dev] gerrit 9904 VIRL verification is failing
Hi,
I sent a patch (https://gerrit.fd.io/r/#/c/9904/) for review in which "vpp-csit-verify-virl-master" job is failing.
Console logs (https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/8798/console.log.gz) shows following error.
====================
call_home\nFlmClientException: Cisco contact was not established. This may be temporary.\nPlease make sure the VIRL server is connected to the Internet and capable of reaching the configured Cisco master.\nAlso make sure that the minion key provided to you matches your minion ID and domain, and remains valid.\nCurrent status is: Last successful contact was more than 7 days ago.\nLast call home check result was: Call has timed out; failed to connect or minion key not accepted.\n"
I have also seen what looks like the same thing on
VIRL3 which is currently not in production. I reported
it yesterday to the CSIT mailing list.
https://jenkins.fd.io/job/csit-vpp-functional-master-ubuntu1604-virl/3290/
}
+ VIRL_SID[${index}]=
+ retval=1
+ '[' 1 -ne 0 ']'
+ echo 'VIRL simulation start failed on 10.30.51.29'
VIRL simulation start failed on 10.30.51.29
=======================
Seems like a temporary problem. What is the gerrit command such that Jenkins again start doing verification.
Thanks,
Nitin
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group
Office of the CTO
Red Hat
|
|
|
Thomas F Herbert <therbert@...>
Ed,
Your fix seemed to fix the problem on VIRL3 when started with
virl-standard-client.
However rechecks are still failing of a load test csit patch.
https://gerrit.fd.io/r/#/c/9900/
--Tom
On 01/02/2018 02:25 PM, Thomas F
Herbert wrote:
Ed,
Thanks for fixing this.
On 01/02/2018 12:51 PM, Ed Kern (ejk)
wrote:
peter was close…the virl salt server we were pointed at did have
a hard crash..
ive repointed all 3 (this also should correct the
issue therbert was seeing on virl3)
at production list of masters and verified all
three as getting good license response now.
Ed, if it due to a limited resource that is invoked VIRL client
starts on all 3 servers, is it possible that the load testing I
was doing on VIRL3 broke it or at least exposed a generic load
testing issue common to all VIRL servers.? I first saw it fail
when I rechecked 3 close to simultaneous rechecks on VIRL3 and a
short time after all rechecks failed. Before It failed,
Let me know if you see issues on your rechecks and
ill give it a harder reset swat.
Ed
On Jan 2, 2018, at 10:04 AM, Marek Gradzki
-X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) < mgradzki@...>
wrote:
+ hc2vpp list (hc2vpp csit
jobs are failing with the same error message)
Regards,
Marek
Hello Ed,
Can you please take a look? Looks like
VIRLs are trying to connect to Cisco master
machines but communication failing (due to company
shutdown?)?
Can you please closer describe why this
connection is required?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems Limited
On 12/27/2017 09:02
AM, Neale Ranns (nranns) wrote:
Hi Nitin,
Hit the ‘reply’ button and post a review comment of:
recheck
that will poke Jenkins to redo the verification.
/neale
-----Original Message-----
From: <vpp-dev-bounces@...> on behalf of "Saxena, Nitin" <Nitin.Saxena@...>
Date: Wednesday, 27 December 2017 at 14:38
To: "Dave Barach (dbarach)" <dbarach@...>, "vpp-dev@..." <vpp-dev@...>
Subject: [vpp-dev] gerrit 9904 VIRL verification is failing
Hi,
I sent a patch (https://gerrit.fd.io/r/#/c/9904/) for review in which "vpp-csit-verify-virl-master" job is failing.
Console logs (https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/8798/console.log.gz) shows following error.
====================
call_home\nFlmClientException: Cisco contact was not established. This may be temporary.\nPlease make sure the VIRL server is connected to the Internet and capable of reaching the configured Cisco master.\nAlso make sure that the minion key provided to you matches your minion ID and domain, and remains valid.\nCurrent status is: Last successful contact was more than 7 days ago.\nLast call home check result was: Call has timed out; failed to connect or minion key not accepted.\n"
I have also seen what
looks like the same thing on VIRL3 which is
currently not in production. I reported it yesterday
to the CSIT mailing list.
https://jenkins.fd.io/job/csit-vpp-functional-master-ubuntu1604-virl/3290/
}
+ VIRL_SID[${index}]=
+ retval=1
+ '[' 1 -ne 0 ']'
+ echo 'VIRL simulation start failed on 10.30.51.29'
VIRL simulation start failed on 10.30.51.29
=======================
Seems like a temporary problem. What is the gerrit command such that Jenkins again start doing verification.
Thanks,
Nitin
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group
Office of the CTO
Red Hat
|
|
|
Thomas F Herbert <therbert@...>
Ed,
IT seems we are good now my load-test patch recheck is working.
Do I dare try the triple load test again?
--Tom
On 01/02/2018 02:34 PM, Thomas F
Herbert wrote:
Ed,
Your fix seemed to fix the problem on VIRL3 when started with
virl-standard-client.
However rechecks are still failing of a load test csit patch.
https://gerrit.fd.io/r/#/c/9900/
--Tom
On 01/02/2018 02:25 PM, Thomas F
Herbert wrote:
Ed,
Thanks for fixing this.
On 01/02/2018 12:51 PM, Ed Kern
(ejk) wrote:
peter was close…the virl salt server we were pointed at did
have a hard crash..
ive repointed all 3 (this also should correct
the issue therbert was seeing on virl3)
at production list of masters and verified all
three as getting good license response now.
Ed, if it due to a limited resource that is invoked VIRL client
starts on all 3 servers, is it possible that the load testing I
was doing on VIRL3 broke it or at least exposed a generic load
testing issue common to all VIRL servers.? I first saw it fail
when I rechecked 3 close to simultaneous rechecks on VIRL3 and a
short time after all rechecks failed. Before It failed,
Let me know if you see issues on your rechecks
and ill give it a harder reset swat.
Ed
On Jan 2, 2018, at 10:04 AM, Marek Gradzki
-X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) < mgradzki@...>
wrote:
+ hc2vpp list (hc2vpp csit
jobs are failing with the same error message)
Regards,
Marek
Hello Ed,
Can you please take a look? Looks like
VIRLs are trying to connect to Cisco master
machines but communication failing (due to
company shutdown?)?
Can you please closer describe why this
connection is required?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems Limited
On 12/27/2017
09:02 AM, Neale Ranns (nranns) wrote:
Hi Nitin,
Hit the ‘reply’ button and post a review comment of:
recheck
that will poke Jenkins to redo the verification.
/neale
-----Original Message-----
From: <vpp-dev-bounces@...> on behalf of "Saxena, Nitin" <Nitin.Saxena@...>
Date: Wednesday, 27 December 2017 at 14:38
To: "Dave Barach (dbarach)" <dbarach@...>, "vpp-dev@..." <vpp-dev@...>
Subject: [vpp-dev] gerrit 9904 VIRL verification is failing
Hi,
I sent a patch (https://gerrit.fd.io/r/#/c/9904/) for review in which "vpp-csit-verify-virl-master" job is failing.
Console logs (https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/8798/console.log.gz) shows following error.
====================
call_home\nFlmClientException: Cisco contact was not established. This may be temporary.\nPlease make sure the VIRL server is connected to the Internet and capable of reaching the configured Cisco master.\nAlso make sure that the minion key provided to you matches your minion ID and domain, and remains valid.\nCurrent status is: Last successful contact was more than 7 days ago.\nLast call home check result was: Call has timed out; failed to connect or minion key not accepted.\n"
I have also seen
what looks like the same thing on VIRL3 which is
currently not in production. I reported it
yesterday to the CSIT mailing list.
https://jenkins.fd.io/job/csit-vpp-functional-master-ubuntu1604-virl/3290/
}
+ VIRL_SID[${index}]=
+ retval=1
+ '[' 1 -ne 0 ']'
+ echo 'VIRL simulation start failed on 10.30.51.29'
VIRL simulation start failed on 10.30.51.29
=======================
Seems like a temporary problem. What is the gerrit command such that Jenkins again start doing verification.
Thanks,
Nitin
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group
Office of the CTO
Red Hat
|
|
|
ah ok missed this before i started my recheck…
of course you could/should try again if it should pass..
of course if you want orange=orange to some past test you may want to wait for my 9900 recheck to finish
Ed
On Jan 2, 2018, at 1:41 PM, Thomas F Herbert < therbert@...> wrote:
Ed,
IT seems we are good now my load-test patch recheck is working.
Do I dare try the triple load test again?
--Tom
On 01/02/2018 02:34 PM, Thomas F Herbert wrote:
Ed,
Your fix seemed to fix the problem on VIRL3 when started with virl-standard-client.
However rechecks are still failing of a load test csit patch.
https://gerrit.fd.io/r/#/c/9900/
--Tom
On 01/02/2018 02:25 PM, Thomas F Herbert wrote:
Ed,
Thanks for fixing this.
On 01/02/2018 12:51 PM, Ed Kern (ejk) wrote:
peter was close…the virl salt server we were pointed at did have a hard crash..
ive repointed all 3 (this also should correct the issue therbert was seeing on virl3)
at production list of masters and verified all three as getting good license response now.
Ed, if it due to a limited resource that is invoked VIRL client starts on all 3 servers, is it possible that the load testing I was doing on VIRL3 broke it or at least exposed a generic load testing issue common to all VIRL servers.? I first saw it fail when
I rechecked 3 close to simultaneous rechecks on VIRL3 and a short time after all rechecks failed. Before It failed,
Let me know if you see issues on your rechecks and ill give it a harder reset swat.
Ed
On Jan 2, 2018, at 10:04 AM, Marek Gradzki -X (mgradzki - PANTHEON TECHNOLOGIES at Cisco) < mgradzki@...> wrote:
+ hc2vpp list (hc2vpp csit jobs are failing with the same error message)
Regards,
Marek
Hello Ed,
Can you please take a look? Looks like VIRLs are trying to connect to Cisco master machines but communication failing (due to company shutdown?)?
Can you please closer describe why this connection is required?
Thanks.
Peter Mikus
Engineer – Software
Cisco Systems Limited
On 12/27/2017 09:02 AM, Neale Ranns (nranns) wrote:
Hi Nitin,
Hit the ‘reply’ button and post a review comment of:
recheck
that will poke Jenkins to redo the verification.
/neale
-----Original Message-----
From: <vpp-dev-bounces@...> on behalf of "Saxena, Nitin" <Nitin.Saxena@...>
Date: Wednesday, 27 December 2017 at 14:38
To: "Dave Barach (dbarach)" <dbarach@...>, "vpp-dev@..." <vpp-dev@...>
Subject: [vpp-dev] gerrit 9904 VIRL verification is failing
Hi,
I sent a patch (https://gerrit.fd.io/r/#/c/9904/) for review in which "vpp-csit-verify-virl-master" job is failing.
Console logs (https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/8798/console.log.gz) shows following error.
====================
call_home\nFlmClientException: Cisco contact was not established. This may be temporary.\nPlease make sure the VIRL server is connected to the Internet and capable of reaching the configured Cisco master.\nAlso make sure that the minion key provided to you matches your minion ID and domain, and remains valid.\nCurrent status is: Last successful contact was more than 7 days ago.\nLast call home check result was: Call has timed out; failed to connect or minion key not accepted.\n"
I have also seen what looks like the same thing on VIRL3 which is currently not in production. I reported it yesterday to the CSIT mailing list.
https://jenkins.fd.io/job/csit-vpp-functional-master-ubuntu1604-virl/3290/
}
+ VIRL_SID[${index}]=
+ retval=1
+ '[' 1 -ne 0 ']'
+ echo 'VIRL simulation start failed on 10.30.51.29'
VIRL simulation start failed on 10.30.51.29
=======================
Seems like a temporary problem. What is the gerrit command such that Jenkins again start doing verification.
Thanks,
Nitin
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
_______________________________________________
vpp-dev mailing list
vpp-dev@...
https://lists.fd.io/mailman/listinfo/vpp-dev
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
--
Thomas F Herbert
NFV and Fast Data Planes
Networking Group Office of the CTO
Red Hat
|
|
|
