CODEWITHSUNDEEP
apachemavenantproxyjenkins
ajk8
ajk8

Reputation: 221

Jenkins build fails occasionally during maven install (or update) on slave executor

I'm running some maven builds in Jenkins, using a Master/multi-slave setup. These builds mostly run fine, but every now and again I get this inexplicable error in the console output for the executing slave, and the build dies:

FATAL: Failed to parse /var/lib/jenkins/updates/hudson.tasks.Maven.MavenInstaller into JSON
hudson.util.IOException2: Failed to parse /var/lib/jenkins/updates/hudson.tasks.Maven.MavenInstaller into JSON
    at hudson.model.DownloadService$Downloadable.getData(DownloadService.java:216)
    at hudson.tools.DownloadFromUrlInstaller$DescriptorImpl.getInstallables(DownloadFromUrlInstaller.java:149)
    at hudson.tools.DownloadFromUrlInstaller.getInstallable(DownloadFromUrlInstaller.java:54)
    at hudson.tools.DownloadFromUrlInstaller.performInstallation(DownloadFromUrlInstaller.java:63)
    at hudson.tools.InstallerTranslator.getToolHome(InstallerTranslator.java:61)
    at hudson.tools.ToolLocationNodeProperty.getToolHome(ToolLocationNodeProperty.java:107)
    at hudson.tools.ToolInstallation.translateFor(ToolInstallation.java:150)
    at hudson.tasks.Maven$MavenInstallation.forNode(Maven.java:515)
    at hudson.tasks.Maven.perform(Maven.java:238)
    at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
    at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:700)
    at hudson.model.Build$RunnerImpl.build(Build.java:178)
    at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
    at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:470)
    at hudson.model.Run.run(Run.java:1409)
    at hudson.matrix.MatrixRun.run(MatrixRun.java:146)
    at hudson.model.ResourceController.execute(ResourceController.java:88)
    at hudson.model.Executor.run(Executor.java:238)
Caused by: net.sf.json.JSONException: A JSONObject text must begin with '{' at character 0 of 
    at net.sf.json.util.JSONTokener.syntaxError(JSONTokener.java:512)
    at net.sf.json.JSONObject._fromJSONTokener(JSONObject.java:843)
    at net.sf.json.JSONObject._fromString(JSONObject.java:1064)
    at net.sf.json.JSONObject.fromObject(JSONObject.java:176)
    at net.sf.json.JSONObject.fromObject(JSONObject.java:147)
    at hudson.model.DownloadService$Downloadable.getData(DownloadService.java:213)
    ... 17 more

The home directory for the slave instance is /home/jenkins, as opposed to /var/lib/jenkins, which could explain the error, but nothing (nothing I've found, anyway) explains why jenkins would be trying to access the wrong home directory.

Has anybody else experienced this problem? Any suggestions for where to go debug-wise?

[UPDATE 20120312]

The direct cause of this problem is empty files in the $JENKINS_HOME/updates/ directory on the jenkins master. In this particular case, the master node is trying to parse $JENKINS_HOME/updates/hudson.tasks.Maven.MavenInstaller and send the data to the slave node so that it can update its maven installation. If I manually populate that file with data from another jenkins installation that I have, all processes go through just fine. However, due to a daily(?) update process, the file is replaced with another empty one, and so the problem persists.

This affects all self-installed tools (ant, java, etc.), and results in little side-effects like the version field under jenkins configuration -> maven -> install from apache being a text field instead of a drop-down.

So, the question becomes why are these files being downloaded empty? I can't find any reference to where they're being downloaded from, so I can't directly debug the link. Even if I could, it appears to work from one Jenkins installation, and not from another. Anyone out there have any suggestions?

Upvotes: 1

Views: 1610

Answers (1)

ajk8
ajk8

Reputation: 221

Problem solved! The jenkins installation is fronted by an apache proxy, and it worked like a charm...almost. A minor tweak was required to get it to an error-free state. I had to change:

ProxyPass         /jenkins/  http://localhost:8080/
ProxyPassReverse  /jenkins/  http://localhost:8080/

To:

ProxyPass         /jenkins/  http://localhost:8080/jenkins/
ProxyPassReverse  /jenkins/  http://localhost:8080/jenkins/

And I had to add:

JENKINS_ARGS="--prefix=/jenkins"

To /etc/sysconfig/jenkins. After that adjustment, everything works fine.

As a side note, the thing that tipped me off to the proxy configuration was that every ajax call was being made twice (once to '/...' and once to '/jenkins/...'). At least, every one that succeeded. Once I made the configuration change, the ajax calls were just being made once each, and performance increased as well.

This misconfiguration makes for some weird side-effects, so hopefully this post can help somebody else.

Upvotes: 1

Related Questions