CODEWITHSUNDEEP
linuxbashenvironment-variables
chrx
chrx

Reputation: 2528

envsubst: default values for unset variables

I've got a json file input.json like the following one:

{
  "variable" : "${ENV_VAR}"
}

of course, I can invoke envsubst from bash like the following:

$ export ENV_VAR=myvalue
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "myvalue"
}

Now, I wish I could set default values for variables in the input.json for the case when ENV_VAR is not set, like in the following example which, as unfortunately can be seen in the example below, doesn't work:

$ cat input.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}
$ export ENV_VAR=newvalue
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}
$ unset ENV_VAR
$ envsubst < input.json > output.json
$ cat output.json
{
  "variable" : "${ENV_VAR:=defaultvalue}"
}

What's curious, if I execute the envsubst like in the following example (without involving an input file), it works

$ export ENV_VAR=myvalue
$ echo "value is ${ENV_VAR:=defaultvalue}" | envsubst
value is myvalue
$ unset ENV_VAR
$ echo "value is ${ENV_VAR:=defaultvalue}" | envsubst
value is defaultvalue

Where is the problem with the files?

Upvotes: 21

Views: 26803

Answers (6)

Dick Visser
Dick Visser

Reputation: 111

For this specific use case, and if you use Bash, you can have jq do the quoting for you. In the input.json, remove the quotes:

{
  "variable" : ${ENV_VAR}
}

debian@bookworm:~$ export ENV_VAR="Foo \" Bar"
debian@bookworm:~$ echo $ENV_VAR
Foo " Bar
debian@bookworm:~$ export ENV_VAR="$(jq -R <<< ${ENV_VAR:-Some default value})"
debian@bookworm:~$ envsubst < input.json
{
  "variable" : "Foo \" Bar"
}

This is handy when your variable contains quotes.

Upvotes: 0

Sweavo
Sweavo

Reputation: 43

https://github.com/a8m/envsubst is a valid answer.

Lubo's comment to mash's 6 mar 2020 answer is wrong: this example is single quoted and the shell does not substitute the variables; the other example was double quoted, inviting the shell to interpolate.

I would have commented on that answer but stack overflow's reputation system is strange.

Upvotes: 0

rzr
rzr

Reputation: 48

There's also https://github.com/busyloop/envcat which supports complex templates (so you can not only do default values but also conditions etc.).

Upvotes: 0

Robin479
Robin479

Reputation: 1686

According to man envsubst, envsubst will only ever replace references to environment variables in the form of ${VAR} or $VAR. Special shell features like ${VAR:-default} are not supported. The only thing you could do is to (re)define all variables in the environment of the envsubst invocation and assign local default values, if they are missing:

ENV_VAR="${ENV_VAR:-defaultvalue}" \
OTHER_VAR="${OTHER_VAR:-otherdefault}" \
envsubst < input.json > output.json

Note, that this is actually a single command line split into multiple lines each ending with a line continuation \. The first two lines are variable assignments, that are only effective in the environment of the executed command envsubst in the last line. What's happening is, that the shell will create an environment for the execution of the command (as it would always do). That environment is initially a copy of the current shell environment. Within that new environment ENV_VAR and OTHER_VAR are assigned the values of expanding the expression ${VAR:-default}, which essentially expands to default unless VAR is defined and has a none-empty value. The command envsubst is executed, receiving the file input.json as standard-input and having its standard-output redirected to output.json (both is done by the shell, transparent to the command). After the command execution, the shell deletes the command environment returning to its original environment, i.e. the local variable assignments are no longer effective.

There is no way to define default values from inside the JSON file, unless you implement a program to do so yourself, or use another tool that can to that.

You could do something like the following, but it is NOT RECOMMENDED:

eval echo "$(cat input.json)" > output.json

which will read input.json into a string, and than evaluate the command echo <string> as if it was type literally, which means that any embedded ${VAR:-default} stuff should be expanded by the shell before the string is passed to echo. BUT any other embedded shell feature will be evaluated as well, which poses a HUGE SECURITY RISK.

Upvotes: 32

javaducky
javaducky

Reputation: 59

Blockquote I'm using https://github.com/a8m/envsubst and it has enhancements over the original gettext envsubst that the expressions in the template file supports default values.

Similarly, a Rust variant called 'envsub' is available which also supports the default values. See https://github.com/stephenc/envsub .

Upvotes: 4

mash
mash

Reputation: 4516

I'm using https://github.com/a8m/envsubst and it has enhancements over the original gettext envsubst that the expressions in the template file supports default values.

The example in the README just works.

echo 'welcome $HOME ${USER:=a8m}' | envsubst

Upvotes: 9

Related Questions