CODEWITHSUNDEEP
bashparallel-processinggnu-parallel
Laura
Laura

Reputation: 105

gnu parallel: space interpretation and within process variable editing

I'm having two problems with gnu parallel. Firstly the most interesting:

I have a file in which one line contains two arguments separated by a space. These arguments should be passed to the command together, in a way that the command can recognize them as separate.

i.e.

/path/to/A1 /path/to/A2  
/path/to/B1 /path/to/B2  
/path/to/C1 /path/to/C2

Additionally I have a second variable as an array. I would like parallel to combine all paired arguments from my file abovewith all array values.

I'm almost there, my code is shown below.

parallel  -a $tmpdir/inputfiles.txt $instaldir/ribotagger.pl  \
                    -in {1}         \
                    -region {2}     \
                    -out $exitdir/$folder/ribotag.{2} \
                    ::: ${regions[@]}

In this instance however, parallel interprets {1} not as

/path/to/A1 /path/to/A2

but as

/path/to/A1\ /path/to/A2

Consequently the ribotagger script interprets it as one long argument, causing an immediate halt.

Second problem, I'd like to have the folder parameter differ for every instance of the script that parallel creates. I thought of something like

-out $exitdir/$(echo {1} | cut -d "/" -f 4)/ribotag.{2}

However, as it appears {1} is not recognized within $(stuff) The script requires an output parameter to run.

Upvotes: 4

Views: 1970

Answers (1)

Mark Setchell
Mark Setchell

Reputation: 207445

I think you need this:

parallel --colsep ' ' -a inputfiles.txt echo 1={1} 2={2} 3={3} ::: france germany | cat -vet
1=/path/to/C1 2=/path/to/C2 3=france$
1=/path/to/C1 2=/path/to/C2 3=germany$
1=/path/to/B1 2=/path/to/B2 3=germany$
1=/path/to/B1 2=/path/to/B2 3=france$
1=/path/to/A1 2=/path/to/A2 3=germany$
1=/path/to/A1 2=/path/to/A2 3=france$

For the output file, you may be able to use {#} (which is the job number) to formulate something you like.

Upvotes: 4

Related Questions