CODEWITHSUNDEEP
bashparametersjenkins
Andrey Regentov
Andrey Regentov

Reputation: 3737

Jenkins text parameter - special characters garbled (unwanted variable substitution)

I have a job in Jenkins (under Linux) with build parameter of type "Text". I use the parameter to form contents of a file which is used in build process, with bash "shell execute" step like echo "$TEXTPARAM" > file.

It works quite well if there is general text. But when characters like "$" appears - it behaves strangely.

E. g. text

Some $one and $$two and $$$more bucks and $ bucks $$ that $$$ stand alone and$ after$$ words$$$

is transformed into

Some $one and $two and $-sl bucks and $ bucks $ that $$ stand alone and$ after$ words$$

though I want the text to appear in file just as it appears in input textbox.

Is it a bug in jenkins (so I should post an issue to their tracker) or am I doing something wrong?

UPDATE I suppose that is due to variable substitution done by Jenkins. I.e all $VARNAMEs are substituted by VARNAME values prior to any "shell execute" steps are executed. And this substitution cannot be turned off as for now.

Upvotes: 4

Views: 18016

Answers (3)

hagello
hagello

Reputation: 3255

Expansion with jenkins-cli.jar

It's true, Jenkins expands build variable names like JENKINS_URL or BUILD_NUMBER when they are prefixed by $. However, there are additional transformations—rather unexpected ones—if you use jenkins-cli.jar.

  • carriage return becomes \r (backslash + "r")
  • new line becomes \n (backslash plus + "n")
  • tab becomes \t (backslash plus "t")

Here is the corresponding part of the source code of jenkins-cli.jar.

I do not know any way of escaping or quoting to keep a white space character that is part of the value of a parameter for a Jenkins job when using jenkins-cli.jar

Expansion when using "raw" ssh (without jenkins-cli.jar)

The Jenkins master handles white space, backslashes and quotes on the command line somewhat like a shell:

  • a remains a
  • 'a' becomes a
  • "a b" becomes a b
  • a b is an error, because the command line parser of Jenkins will see b
  • a" " becomes a ("a" plus space)

My idea was to re-implement the code that does the quoting in jenkins-cli.jar (minus the bugs when handling tab characters and the like). So here's my recipe:

For each argument, escape each backslash and each single quote with a backslash. Then surround it with single quotes.

Example: Instead of a'"b, send 'a\'"b'.

This has proven to be protect white space and quotes. And instead of using single quotes, you can use double quotes instead.

Pipeline for Testing

This is how I tested: I created the pipeline test-quoting with the string parameter "PARAM" and the following script:

import groovy.json.JsonOutput
node {
    println JsonOutput.toJson(env.PARAM)
}

Then I started the pipeline (in bash), adding an additional layer of quoting that the local shell will remove:

# for testing 'a b'
$ ssh -x -p 50022 <jenkins-server> -l <user-name> build -s -v test-quoting -p PARAM="'a b'"

# for testing the behaviour of the cli with tab character
$ java -jar jenkins-cli.jar -ssh -s <jenkins-url> -user <user-name> build -s -v test-quoting -p PARAM="$(printf '\t')"

If you are unsure about what your local shell really passes to ssh (or any other command), prefix the command with strace -e execve.

Upvotes: 3

Slav
Slav

Reputation: 27485

This has nothing to do with Jenkins.

Write a bash script with echo "some $$$more" > file and execute that on a linux command prompt, and you will get the same gibberish.

Special characters, like $ must be escaped, since this is a linux environment and $ means a variable. There are several ways to do it.

Option 1.
Use per-character escape, i.e. for every $ that you want to appear literally, use \$ instead. So it becomes:
echo "some \$\$\$more" > file

Option 2.
Use strong-quoting, i.e. single quotes '. Nothing within single quotes has any special meaning, except for a second single quote to close the string:
echo 'some $$$more' > file
Of course with this method, you have to make sure your $TEXTPARAM string does not have any single quotes of it's own.

In either case, you will have to sanitize your input. Before you output it to file, you will need to parse the content of $TEXTPARAM and either replace all $ with \$ and use Option 1. Or parse your $TEXTPARAM and remove all single quotes ' before outputting that to file using Option 2.

Edit

In your case, I think you just want:
echo $TEXTPARAM > file without any extra quotes

root@ ~ $ cat test.sh
#!/bin/bash
TEXTPARAM='Some $one and $$two and $$$more bucks and $ bucks $$ that $$$ stand alone and$ after$$ words$$$'
echo $TEXTPARAM
echo $TEXTPARAM > file
cat file
root@ ~ $ ./test.sh
Some $one and $$two and $$$more bucks and $ bucks $$ that $$$ stand alone and$ after$$ words$$$
Some $one and $$two and $$$more bucks and $ bucks $$ that $$$ stand alone and$ after$$ words$$$

Upvotes: -1

dgeorgiev
dgeorgiev

Reputation: 941

According to a comment in this ticket https://issues.jenkins-ci.org/browse/JENKINS-16143

This appears to not be a bug. Compare the parameter values $JENKINS_URL and $$JENKINS_URL. Jenkins internally resolves placeholders in variables, and dollars are used for that. $$ is an escaped $.

I am observing the same behavior for string and text fields on Jenkins ver. 1.562

Upvotes: 2

Related Questions