CODEWITHSUNDEEP
serializationjenkinsgroovyjenkins-pipeline
Henok
Henok

Reputation: 423

Jenkins 2 Pipeline Serialization problems in global function

I defined a global function in the vars directory of my Global Pipeline Library. I always got java.io.NotSerializableException caused by JsonSlurper. I replaced it by JsonSlurperClassic and now I get a similar execption caused by HttpURLConnection. I could not find any documentaion how I could handle serialization issues in global functions. @NonCPS does not work. I created a file names sonar.groovy in the vars directory and added the following code:

import groovy.json.JsonSlurperClassic

def call(Map params) {

    // sonar maven scanner
    def sonarCmd = 'org.sonarsource.scanner.maven:sonar-maven-plugin:3.2:sonar'

    // set default value if a param is null
    params.sonarId = params.sonarId ?: 'Sonar'
    params.dir = params.dir ?: '.'
    params.failOnError = params.failOnError ?: true

    withSonarQubeEnv('Sonar-Dev') {
        maven(dir: params.dir, cmd: sonarCmd)
    }

    // parse analysis status and throw an error if the analysis does not pass the quality gate
    if (params.failOnError) {
        def jsonParser = { url -> new JsonSlurperClassic().parse(url.openStream()) }

        def timeoutInMinutes = 10

        def props = readProperties file: "${params.dir}/target/sonar/report-task.txt"

        def ceTaskUrl = new URL(props.ceTaskUrl)
        def ceTask

        timeout(time: timeoutInMinutes, unit: 'MINUTES') {
            waitUntil {
                ceTask = jsonParser(ceTaskUrl)
                echo ceTask.toString()
                return "SUCCESS" == ceTask.task.status
            }
        }

        def analysisId = ceTask.task.analysisId

        def statusUrl = new URL("${props.serverUrl}/api/qualitygates/project_status?analysisId=${analysisId}")

        def analysisStatusJson = jsonParser(statusUrl)

        def status = analysisStatusJson.projectStatus.status

        if (status == "ERROR") {
            error "Sonar Analysis: Quality Gate failure"
        }
    }
}

I always get this error:

[Pipeline] }
[Pipeline] // wrap
[Pipeline] readProperties
[Pipeline] timeout
Timeout set to expire in 10 min
[Pipeline] {
[Pipeline] waitUntil
[Pipeline] {
[Pipeline] }
[Pipeline] // waitUntil
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
java.io.NotSerializableException: sun.net.www.protocol.http.HttpURLConnection$HttpInputStream
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:860)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:778)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:967)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.BlockMarshaller.doWriteObject(BlockMarshaller.java:65)
    at org.jboss.marshalling.river.BlockMarshaller.writeObject(BlockMarshaller.java:56)
    at org.jboss.marshalling.MarshallerObjectOutputStream.writeObjectOverride(MarshallerObjectOutputStream.java:50)
    at org.jboss.marshalling.river.RiverObjectOutputStream.writeObjectOverride(RiverObjectOutputStream.java:179)
    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:344)
    at java.util.HashMap.internalWriteEntries(HashMap.java:1785)
    at java.util.HashMap.writeObject(HashMap.java:1362)
    at sun.reflect.GeneratedMethodAccessor263.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.jboss.marshalling.reflect.SerializableClass.callWriteObject(SerializableClass.java:271)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:976)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteFields(RiverMarshaller.java:1032)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteSerializableObject(RiverMarshaller.java:988)
    at org.jboss.marshalling.river.RiverMarshaller.doWriteObject(RiverMarshaller.java:854)
    at org.jboss.marshalling.AbstractObjectOutput.writeObject(AbstractObjectOutput.java:58)
    at org.jboss.marshalling.AbstractMarshaller.writeObject(AbstractMarshaller.java:111)
    at org.jenkinsci.plugins.workflow.support.pickles.serialization.RiverWriter.writeObject(RiverWriter.java:132)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:452)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgram(CpsThreadGroup.java:427)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.saveProgramIfPossible(CpsThreadGroup.java:415)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.run(CpsThreadGroup.java:360)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup.access$100(CpsThreadGroup.java:80)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:240)
    at org.jenkinsci.plugins.workflow.cps.CpsThreadGroup$2.call(CpsThreadGroup.java:228)
    at org.jenkinsci.plugins.workflow.cps.CpsVmExecutorService$2.call(CpsVmExecutorService.java:64)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at hudson.remoting.SingleLaneExecutorService$1.run(SingleLaneExecutorService.java:112)
    at jenkins.util.ContextResettingExecutorService$1.run(ContextResettingExecutorService.java:28)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
Caused by: an exception which occurred:
    in field arguments
    in field abnormal
    in field outcome
    in field body
    in field step
    in field thread
    in field body
    in field step
    in field thread
    in field this$0
    in field returnAddress
    in field parent
    in field caller
    in field parent
    in field parent
    in field caller
    in field parent
    in field parent
    in field parent
    in field capture
    in field def
    in field closures
    in object org.jenkinsci.plugins.workflow.cps.CpsThreadGroup@54c4acb
Finished: FAILURE

Is there any way to handle serialization problem in a global function?

Upvotes: 2

Views: 4881

Answers (2)

Jon S
Jon S

Reputation: 16346

Good that you got it resolved! Some additional comments and way of solving this. All variables that are defined in the Jenkins pipeline must be serializable, reason for this is that Jenkins may pause an ongoing pipeline execution at any point and serialize the state. So, that is why you get the error.

The general way to solve this is to wrap the unserializable code inside a function annotated with @NonCPS. This tells Jenkins that the function contains unserializable parts and must be executed without pausing.

More info can be found here https://github.com/jenkinsci/pipeline-plugin/blob/master/TUTORIAL.md#serializing-local-variables

Upvotes: 3

Henok
Henok

Reputation: 423

I solved the problem by replacing openStream() with newReader()

def jsonParser = { url -> new JsonSlurperClassic().parse(url.newReader()) }

Upvotes: 1

Related Questions