CODEWITHSUNDEEP
bashparallel-processinggnu-parallel
Frank-Rene Schäfer
Frank-Rene Schäfer

Reputation: 3352

GNU Parallel: how to prevent specific jobs from being processed in parallel

GIVEN:

A set of jobs to be run in parallel: { app0, app1, app2, .... }

QUESTION:

How is it possible to initiate the tool 'GNU parallel' to run all jobs in parallel, whereby some specific jobs prevented from running concurrently?

EXAMPLE:

If appX and appY rely on the same resources, how can one specify that appX may run in parallel with app0, app1, ... but never with appY?

EXAMPLE 2:

appX and appY may run in parallel, but neither of them shall be running concurrently with appZ.

Upvotes: 2

Views: 197

Answers (1)

Ole Tange
Ole Tange

Reputation: 33715

It is not 100% clear to me what you want. Maybe replace appX and appY with:

sem --id myidXY --fg appX
sem --id myidXY --fg appY

Which can be done like this:

... | parallel eval '{= s/(app(X|Y))/sem --id appXY --fg $1/ =}'

This should make sure only a single appX or appY is running; but let plenty of appZs running.

{= =} is interpreted as Perl code.

s/(app(X|Y))/sem --id appXY --fg $1/ replaces appX or appY with sem --id appXY --fg followed by either appX or appY depending in what was matched. If nothing is matched then the value is unchanged.

(echo appX; echo appX; echo appX; 
 echo appY; echo appX; echo appV;
 echo appX; echo appZ) |
  parallel eval '{= s/(app(X|Y))/sem --id appXY $1/ =}'

If that is not what you mean, please edit the question.

Upvotes: 2

Related Questions