CODEWITHSUNDEEP
parallel-processinggnu-parallel
Sj L
Sj L

Reputation: 41

How to use gnu_parallel to run an executable from a list of files?

I am currently learning to use the gnu_paralle. I have read https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Working-as-xargs--n1.-Argument-appending

and also a few other resources and had some success learning the syntax.

At this moment, I have a g++ compiled executable which takes in an *argv. I currently run it in series by doing the following in bash:

cd ./ && find ./data | grep '\.data.format$' > datalist.tmp;

I then read this list of files in the following order

while read LINE; do executable $LINE; done <datalist.tmp

Now. From reading gnu_parallel documentations I understood that processing the executable in multiple instances can be paralleled.

Will someone help with the syntax here please?

Upvotes: 1

Views: 117

Answers (2)

Sj L
Sj L

Reputation: 41

Thank you. I ended up finding the following solution,

while read LINE; do echo executable.exe $LINE ; done < data.tmp |paralell -j4

Upvotes: 0

Mark Setchell
Mark Setchell

Reputation: 207425

Try:

parallel -a datalist.tmp executable

Or:

parallel executable < datalist.tmp

Not sure what cd ./ is doing - it appears to change directory to the current one, which seems pointless because that's where the process is already.

Upvotes: 1

Related Questions