CODEWITHSUNDEEP
bashparallel-processing
user2804865
user2804865

Reputation: 1064

bash: running script on multiple directories simultaneously

Let's say that I have multiple directories: nmg_1, nmg_2,..., nmg_5. Each of these directories contains two subdirectories: fol, unf. In each of the subdirectories there is a text file.

So, I have this script: run.sh, that is to read a certain line of the text files in each subdirectory and output another file. Thus, I was wondering, is there a way to run the script in parallel, that is, for the script to be run in multiple subdirectories at once?

Upvotes: 1

Views: 572

Answers (1)

Eugeniu Rosca
Eugeniu Rosca

Reputation: 5305

Given run.sh which does the required processing on one text file at a time, the following command will keep 4 parallel bash sessions alive:

find -name "*.txt" -print0 | xargs -n 1 -P 4 -0 -I {} bash run.sh {}

From man xargs:

--max-procs=max-procs, -P max-procs
      Run up to max-procs processes at a time; the default is 1.
      If max-procs is 0, xargs will run as many processes as possible at a time. 
      Use the -n option with -P; otherwise chances are that only one exec will be done.

Upvotes: 3

Related Questions