Itai Ganot
Itai Ganot

Reputation: 6305

Jenkins pipeline: Running a shell command returns "Bad substitution", but why?

I'd like to populate the the groovy variable "committer" with the output of the command:

def committer = utils.sh("curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json | python -mjson.tool | grep authorEmail | awk '{print \$2}' | tr -d '"|,' ")

Because of the known issue in Jenkins (JENKINS-26133) it is not possible to do that but only to populate the variable with the exit status of the command.

So I've go these 2 functions:

def gen_uuid(){
    randomUUID() as String
}

def sh_out(cmd){ // As required by bug JENKINS-26133
    String uuid = gen_uuid()
    sh """( ${cmd} )> ${uuid}"""
    String out = readFile(uuid).trim()
    sh "set +x ; rm ${uuid}"
    return out
}

These functions allow me to wrap my shell commands in sh_out(COMMAND) and in the background I'm using the workaround which is suggested in the mentioned above known issue link which means running the command while redirecting it's output to a file (in the case of my function it's a random filename) and then reading it into a variable.

So, In the beginning of my pipeline I load my functions file which ends with return this; like so:

fileLoader.withGit('[email protected]:company/pipeline_utils.git', 'master', git_creds, ''){
    utils = fileLoader.load('functions.groovy');
}

And that's why the "utils.sh_out" that you see in the command, but when I use the shown above command in my Jenkins pipeline, I get the following error:

/home/ubuntu/workspace/-6870-bitbucket-integration-ECOPKSSBUJ6HCDNM4TOY77X7UTZ@tmp/durable-006d5c7e/script.sh: 2: /home/ubuntu/workspace/-6870-bitbucket-integration-ECOPKSSBUJ6HCDNM4TOY77X7UTZ@tmp/durable-006d5c7e/script.sh: Bad substitution

Running the command in a shell works properly:

$ curl -s -u user:password http://IPADDR:8080/job/COMPANY_BitBucket_Integration/job/research/job/COMPANY-6870-bitbucket-integration/3/api/json/api/json | python -mjson.tool | grep authorEmail | awk '{print $2}' | tr -d '"|,'
[email protected]

I suspect it has something to do with the tr command in the end and with the character escaping I did there but whatever I try fails, anyone got an idea?

Upvotes: 1

Views: 4178

Answers (1)

daggett
daggett

Reputation: 28564

according to the documentation now sh supports std output.

and i know i'm not answering your question directly, but i suggest to use groovy to parse json.

you are trying to get the value of authorEmail from json

if the response from /api/json looks like this (just an example):

{
  "a":{
    "b":{
      "c":"ccc",
      "authorEmail":"[email protected]"
    }
  }
}

then the groovy to take athorEmail:

def cmd = "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json"
def json = sh(returnStdout: true, script: cmd).trim()
//parse json and access it as an object (Map/Array)
json = new groovy.json.JsonSlurper().parseText(json)
def mail = json.a.b.athorEmail

you could receive java.io.NotSerializableException explained here

so i changed the code like this:

node {
    def json = sh(
            returnStdout: true, 
            script: "curl -s -u \${J_USER}:\${J_PASS} \${env.BUILD_URL}/api/json"
        ).trim()
    def mail = evaluateJson(json, '${json.a.b.authorEmail}')
    echo mail
}

@NonCPS
def evaluateJson(String json, String gpath){
    //parse json
    def ojson = new groovy.json.JsonSlurper().parseText(json)
    //evaluate gpath as a gstring template where $json is a parsed json parameter
    return new groovy.text.GStringTemplateEngine().createTemplate(gpath).make(json:ojson).toString()
}

Upvotes: 1

Related Questions