CODEWITHSUNDEEP
matlabparallel-processing
David Parks
David Parks

Reputation: 32111

Matlab parallel processing: Can I limit the number of workers used at the parfor level?

I want to know if it's possible to specify the max # of workers at the parfor level?

I know how to change the # of workers in a matlab parallel pool, but I want to change the workers assigned to a parfor loop quickly and dynmically.

Rational: I have some long running parfor loops and sometimes an odd number of jobs, say 5, will leave one waiting for the 4 others to finish before the last one starts, and this is very slow compared to just running all 5 in parallel. What I'd like is a little bit of code that can determine if there are 5 jobs, run all 5 in parallel, but if there are 6, maybe running 2 sets of 3 jobs is a better setup, hence we use a limit of 3 workers for this case, but 6 for the former.

Shutting down and restarting the parallel pool to achieve this is rather slow and annoying. If there's a way to just startup with, say, 10 workers and dynamically decide how many is most efficient to use at runtime, that would solve the problem.

Upvotes: 0

Views: 1839

Answers (1)

Amro
Amro

Reputation: 124553

According to the documentation, there is a second syntax for parfor:

parfor (loopvar = initval:endval, M)
    statements
end

This executes statements inside the loop using a maximum of M workers:

uses M to specify the maximum number of MATLAB workers that will evaluate statements in the body of the parfor-loop. M must be a nonnegative integer. By default, MATLAB uses as many workers as it finds available. If you specify an upper limit, MATLAB employs no more than that number, even if additional workers are available. If you request more resources than are available, MATLAB uses the maximum number available at the time of the call.

Upvotes: 3

Related Questions