CODEWITHSUNDEEP
bashgnu-parallel
Florian van Kampen-Wright
Florian van Kampen-Wright

Reputation: 443

How do you send commands with arguments already inside them to GNU parallel?

I have a bash array:

nodes=(
    "command"
    "command arg"
    ...
    )

and I want to run all the commands with all the arguments that are already attached to them using GNU parallel.

I've tried

printf '%s\n' "${nodes[@]}" | parallel python

and 

parallel python ::: "${nodes[@]}"

The output command is

python path_to_file\ arg

and the error it gives is "can't open file 'path_to_file arg'"

I think the problem has to do that backslash - I get the same error when I run the command without parallel.

How do I prevent it from putting the backslash in?

Upvotes: 2

Views: 461

Answers (2)

Ole Tange
Ole Tange

Reputation: 33748

Try:

printf '%s\n' "${nodes[@]}" | parallel eval python

eval is a shell command that evals the string as shell expression. I typically used it to "de-quote" a string.

or:

printf '%s\n' "${nodes[@]}" | parallel python {=uq=}

Newer versions of GNU Parallel have uq() which leaves the value unquoted. Normally GNU Parallel will quote values.

or:

printf '%s\n' "${nodes[@]}" | parallel

The exception to the rule above is when there is no command. Then the value is unquoted.

Upvotes: 2

KamilCuk
KamilCuk

Reputation: 141920

You need to split you input on spaces into arguments. You can for example (ab-)use word splitting expansion by the shell. parallel passed your arguments as they are, so you run python "command arg" - with one argument with literal space preserved. Yet I guess you want python command arg - pass two arguments and the space acts as a separator.

parallel 'python $*' ::: "${nodes[@]}"

Using xargs that would look like:

printf "%s\n" "${nodes[@]}" | xargs -l sh -c 'python $*' --

Upvotes: 1

Related Questions