[honeycomb-dev] [FD.io Helpdesk #30789] Honeycomb merge jobs are failing

Thanh Ha via RT fdio-helpdesk at rt.linuxfoundation.org
Tue Oct 11 21:23:58 UTC 2016

On Wed Sep 28 07:49:52 2016, mgradzki at cisco.com wrote:
> Hi,
> HC merge jobs are failing on ubuntu14. It looks like wrong java
> version is used for sonar:
> https://jenkins.fd.io/job/honeycomb-merge-master-
> ubuntu1404/1133/console
> could you please take a look?
> Regards,
> Marek

Hi Everyone,

I started looking at this today and took quite a bit longer to triage but the issue is now resolved. I will try to explain in details below.


Sonar runs against Java 7 instead of Java 8

This is because mvn command uses JAVA_HOME variable to determine the Java version it runs against but since the Jobs do not declare a Java version Jenkins will default to system level. Unfortunately this is undeterministic as we cannot know what version of java is being pulled in from what scripts and where.

Attempts to fix this with patches:


Failed because Maven does not use the default Java declared by the OS. It only uses JAVA_HOME declaration which we cannot be sure what's being passed in by Jenkins until we explicitly set it. Attempts to manipulate the build via shell scripts were futile as Jenkins does not take it's input from the shell scripts.

Additionally Jenkins 2 which fd.io is using had moved the JDK global configuration to a new location. This declaration is necessary to inform Jenkins where to look for explicit versions of the JDK at the file system level such that we can at the job level define which JDK should be used during a job run.


The solution is 2 fold.

1) Configure JDK names and locations in Jenkins Global Configuration.

In Jenkins 1 this was available in /configure in Jenkins but has sinced been moved in Jenkins 2 to Manage Jenkins > Global Tools Configuration page. We've added JDK declarations as follows:

- openjdk7-centos7
- openjdk8-centos7
- openjdk7-ubuntu1404
- openjdk8-ubuntu1404
- openjdk7-ubuntu1604
- openjdk8-ubuntu1604

2) Reconfigure honeycomb jobs to explicitly specify which JDK should be used to build

We contributed this patch https://gerrit.fd.io/r/3360 to resolve this for honeycomb-merge jobs. The jobs now have an additional "jdk: openjdk8-{os}" configuration to handle the different OS versions and JDK matching.

Moving forward:

While this fixes the honeycomb-merge jobs there are additional steps that should be taken for all other jobs in fd.io that are Java related. We need to make similar modifications to 2) above for every Java build in fd.io to explicitly configure the JDK used in the build so that we don't rely on file system level defaults as this affects the health of a job.


More information about the honeycomb-dev mailing list