CODEWITHSUNDEEP
matlabparallel-processingload-balancing
user1500849
user1500849

Reputation: 1

Matlab parfor work distribution

I have a parfor loop through say 100 iterations, and the workload on every iteration is different but changes linearly in a way that the first one takes the most time and the last one is the fastest. But when I run through the parfor loop with my four instances/labs, during the last few hours only one lab is active as it's running through the few first iterations by its own.

So I know which iterations are the slow ones. How could I make workload between cores more even. For example somehow force all labs to start working on the first four slow ones and then proceed in order? Or something similar to prevent only one active core running the few slow ones alone..

Upvotes: 0

Views: 1513

Answers (2)

Edric
Edric

Reputation: 25160

I don't think this is documented anywhere, but you can quickly deduce that PARFOR runs iterations in reverse loop order (using pause and disp if you want to see it in action). So, you should simply reverse your loop. PARFOR gives you no means to explicitly control execution order, but SPMD using for-drange does (PARFOR is significantly easier to use though).

@denahiro's suggestion is also a good one.

Upvotes: 1

denahiro
denahiro

Reputation: 1199

Matlab parfor does nothing more but split up the indices and distributes them to the workers. It does this by creating contiguous chunks from the indices. I don't know the exact algorithm but this means that data with similar indices get computed in the same chunk and by the same worker.

The simplest solution would be a stochastic one. Just shuffle your indices so that the work intensive steps are distributed nicely. While this doesn't give you any guarantees on performance it is simple and will work most of the time.

Some example code:

% dummy data
N=10;
data=1:N;

% generate the permutated indices
permIndex=randperm(N);

% permute the data
dataPermuted=data(permIndex);

% run the loop
parfor i=1:N
    % do something e.g. pause for the time as specified by data
    pause(dataPermuted(i));
end

%invert the index permutation
dataInversePermuted(permIndex)=dataPermuted;

I used pause to simulate the different computation times.

Upvotes: 2

Related Questions