CODEWITHSUNDEEP
zshgnu-parallel
florit
florit

Reputation: 345

Escape variables containing spaces and special characters in GNU parallel

I try to parallelize the optimization of a PDF file. (I’m on a Mac)

#!/bin/zsh

TMP_DIR=$(mktemp -d)
DOCUMENT="/some/path/with/sp aces/and/üö chars.pdf"

mkdir "$DOCUMENT"_split

#split pdf into single pages
/usr/local/bin/pdfseparate "$DOCUMENT" "$TMP_DIR/${$(basename $DOCUMENT)%.pdf}_%d.pdf"

find "$TMP_DIR" -mindepth 1 -maxdepth 1 -name "*.pdf" ! -print0 | parallel -0 -j+0 '/usr/local/bin/ps2pdf {} {.}_optimized.pdf && mv {.}_optimized.pdf $DOCUMENT_split/$(basename {});'

Everything works fine, as long as there are no spaces or special characters in the path. The mvcommand fails:

usage: mv [-f | -i | -n] [-v] source target
       mv [-f | -i | -n] [-v] source ... directory

I tried the following which helps the directory path, but wrapping $(basename {}) the same way doesn’t work.

mv {.}_optimized.pdf '\"$DOCUMENT\"'_split/$(basename {})

Upvotes: 2

Views: 777

Answers (2)

Ole Tange
Ole Tange

Reputation: 33685

GNU Parallel's replacement strings will quote the resulting string correctly. This means that it is safe to use on even crazy file names:

touch "  Spacey  My brother's 12\" records.txt"
find . -print0 | parallel -0 echo {} {.}

This guarantee does not apply to variables and especially not to output from commands executed in the command template.

# This does not do what you expect
DOCUMENT="It's  \"two\"  spaces"
find . -print0 | parallel -0 echo $DOCUMENT $(basename {})

Instead of basename you can use the replacement string {.}. $DOCUMENT is harder to get right every time:

DOCUMENT="It's  \"two\"  spaces"
export DOCUMENT
find . -print0 | parallel -0 echo '"$DOCUMENT"' {.}

Often it is easier just to make a bash function and call that:

doit() {
  f="$1"
  echo "$DOCUMENT" "$(basename "$f")"
}
export -f doit
export DOCUMENT
find . -print0 | parallel -0 doit

(PS: -j+0 has been default for years).

Upvotes: 1

user1934428
user1934428

Reputation: 22225

It just occured to me that there is a much easier possibility than fiddling around with all that quoting: If you make DOCUMENT an environment variable, you can let do all the expansion by the shell invoked by parallel:

export DOCUMENT

Only since I don't know what shell parallel is using, I would not rely on the hope that it runs zsh, and would prepare the argument so that it would work even in a POSIX shell:

... parallel -0 -j+0 '/usr/local/bin/ps2pdf {} {.}_optimized.pdf && mv {.}_optimized.pdf "$DOCUMENT_split/$(basename "{}")";'

In this way, the shell executed by parallel sees exactly what is in between the two single quotes. This means that you even can test that approach by substituting suitable values for DOCUMENT and {}.

Upvotes: 0

Related Questions