CODEWITHSUNDEEP
bashvariablesquoting
fresskoma
fresskoma

Reputation: 25791

Using a string which contains quotes as parameter to a command in bash without additional escaping

Background

What I'm trying to do is exclude all submodules of a git repository in a find command. I know that I can exclude a single directory like this:

find . -not -path './$submodule/*'

So I built a command that generates a lot of these statements and stores them:

EXCLUDES=$(for submodule in $(git submodule status | awk '{print $2}'); do
    echo -n "-not -path './$submodule/*' ";
done)

Problem

But when I run find . $EXCLUDES, this does not work. I suspect this is because of a bash quoting strategy that I do not understand. For example, lets assume (# marks output):

tree .
# .
# ├── bar
# │   └── baz.scala
# └── foo.scala

set -x
EXCLUDES="-not -path './bar/*'"
find . -type f $EXCLUDES
# + find . -not -path ''\''./bar/*'\''' <---- weird stuff
# foo.scala
# baz.scala

find . -type f -not -path './bar/*'
# + find . -type f -not -path './bar/*'
# foo.scala

How do I tell bash not to to the weird quoting stuff its doing (see marked line above)?

Edit: @eddiem suggested using git ls-files, which I will do in this concrete case. But I'm still interested in how I'd do this in the general case where I have a variable with quotes and would like to use it as arguments to a command.

Upvotes: 2

Views: 119

Answers (1)

cxw
cxw

Reputation: 17051

The "weird stuff" you note is because bash only expands $EXCLUDES once, by substituting in the value you stored in EXCLUDES. It does not recursively process the contents of EXCLUDES to remove single-quotes like it does when you specify the quoted string on the command line. Instead, bash escapes special characters in $EXCLUDES, assuming that you want them there:

-not -path './bar/*'

becomes

-not -path ''\''./bar/*'\'''
             ^^         ^^ escaped single quotes
           ^^             ^^ random empty strings I'm actually not sure about
               ^       ^ single quotes around the rest of your text.

So, as @Jean-FrançoisFabre said, if you leave off the single quotes in EXCLUDES=..., you won't get the weird stuff.

So why isn't the first find working as expected? Because bash expands $EXCLUDES into a single word, i.e., a single element of argv that gets passed to find.* However, find expects its arguments to be separate words. As a result, find does not do what you expect.

The most reliable way I know of to do this sort of thing is to use an array:

declare -a EXCLUDES    #make a new array
EXCLUDES+=("-not" "-path" './bar/*')
    # single-quotes       ^       ^ so we don't glob when creating the array

and you can repeat the += line any number of times for exclusions that you want. Then, to use these:

find . -type f "${EXCLUDES[@]}"

The "${name[@]}" form, with all that punctuation, expands each element of the array to a separate word, but does not further expand those words. So ./bar/* will stay as that and not be globbed. (If you do want globbing, find . -type f ${EXCLUDES[@]} (without the "") will expand each element of the array.)

Edit By the way, to see what's in your array, do set|grep EXCLUDES. You will each each element listed separately. You can also do echo "${EXCLUDES[@]}", but I find that less useful for debugging since it doesn't show the indices.

* see the "expansion" section of the man page. "Parameter expansion," expanding things that start with $, cannot change the number of words on the command line — except for "$@" and "${name[@]}".

Upvotes: 2

Related Questions