CODEWITHSUNDEEP
bashparallel-processinggnu-parallel
Frank-Rene Schäfer
Frank-Rene Schäfer

Reputation: 3352

GNU Parallel: Giving a hint about execution time

GIVEN:

Some very much time-consuming jobs, BIG0, BIG1, etc. and a bunch of small fast jobs fast0, fast1, etc.

QUESTION:

Assume, that the BIG* jobs take 2 hours, and the fast* jobs take 1 hour. How can I instruct GNU Parallel to distribute the jobs over time and cores as follows

   time[h]    CPU[0]  CPU[1]     
     0        BIG0    BIG1     
     1         :        :       
     2        fast0   fast1

and not as shown below?

   time[h]   CPU[0]   CPU[1] 
     0       BIG0     fast0   
     1         :      fast1
     2       BIG1            
     3         :

The last setup would take one hour longer.

Upvotes: 2

Views: 229

Answers (1)

Mark Setchell
Mark Setchell

Reputation: 207738

One possible approach might be to sort your jobs by expected time, before submitting to GNU Parallel, big ones first:

parallel ... ::: BIG* fast*

Upvotes: 1

Related Questions