CODEWITHSUNDEEP
bash
leafo
leafo

Reputation: 1852

Why does command substitution change how quoted arguments work?

I have the following snippet:

printf "%s\n%s\n%s" $(echo "'hello world' world")

I would expect it to produce:

hello world
world

But it actually produces:

'hello
world'
world

Why is the above command not the same as the following?

printf "%s\n%s\n%s" 'hello world' world

Upvotes: 10

Views: 967

Answers (4)

Gringo Suave
Gringo Suave

Reputation: 31870

Lots of discussion why, but no one has bothered to add how to work around this behavior?

So far I've found running eval $(...) to reevaluate the quotes works. Though of course use this sparingly, and not at all if you can't trust the output of the command substitution.

Upvotes: 1

Todd A. Jacobs
Todd A. Jacobs

Reputation: 84343

TL;DR

Bash doesn't see what you think it should see because Bash splits words after command substitution.

What Bash Sees After Word-Splitting

Your two code samples are not equivalent. In your non-substitution example, there is no command substitution performed, so the quoting from step 2 of shell operation will cause your text to be seen as two arguments. For example:

$ set -x
$ printf "%s\n%s\n%s" 'hello world' world
+ printf '%s\n%s\n%s' 'hello world' world
hello world
world

In your other example, Bash doesn't reinterpret the quoting characters that remain after command substitution, but does perform word-splitting after performing shell expansions. For example:

$ set -x
$ printf "%s\n%s\n%s" $(echo "'hello world' world")
++ echo ''\''hello world'\'' world'
+ printf '%s\n%s\n%s' ''\''hello' 'world'\''' world
'hello
world'

So, Bash sees ''\''hello' 'world'\''' world and then splits the words according to $IFS. This leaves literal single-quotes in the first two words, and the last word is not used by your printf statement. An easier way to see this is to change the last word or remove the excess quotes:

# The third argument is never used.
$ printf "%s\n%s\n%s" $(echo "'hello world' unused_word")
'hello
world'

# Still breaks into three words, but no quote literals are left behind!
$ printf "%s\n%s\n%s" $(echo 'hello world' unused_word)
hello
world

Upvotes: 3

Barmar
Barmar

Reputation: 780818

After command substitution, only word splitting and wildcard expansion are done, but not quote processing.

From tbe Bash Reference Manual

The order of expansions is: brace expansion, tilde expansion, parameter, variable, and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and filename expansion.

The description of word splitting is:

The shell scans the results of parameter expansion, command substitution, and arithmetic expansion that did not occur within double quotes for word splitting.

The shell treats each character of $IFS as a delimiter, and splits the results of the other expansions into words on these characters. If IFS is unset, or its value is exactly <space><tab><newline>, the default, then sequences of <space>, <tab>, and <newline> at the beginning and end of the results of the previous expansions are ignored, and any sequence of IFS characters not at the beginning or end serves to delimit words. If IFS has a value other than the default, then sequences of the whitespace characters space and tab are ignored at the beginning and end of the word, as long as the whitespace character is in the value of IFS (an IFS whitespace character). Any character in IFS that is not IFS whitespace, along with any adjacent IFS whitespace characters, delimits a field. A sequence of IFS whitespace characters is also treated as a delimiter. If the value of IFS is null, no word splitting occurs.

No mention is made of treating quotes specially when doing this.

Upvotes: 5

kamituel
kamituel

Reputation: 35950

The following:

$(echo "'hello world' world")

outputs three tokens: 'hello, world', world. Those three tokens are being passed to the printf this way:

printf "%s\n%s\n%s" ('hello) (world') (world)

and not this way:

 printf "%s\n%s\n%s" ('hello world') (world)

(note that parentheses are here only to visually separate tokens, to illustrate the point).

Upvotes: 1

Related Questions