Graph4Me Consultant
Graph4Me Consultant

Reputation: 353

Parallel computation using big matrix in Matlab

I have a big matrix M (size like 18000 x 18000) and an array k of size

k =[k_1,..,k_{100*50}]

a small matrix C (like 18000 x 100)

and an iterator

iIterator = [1,...,100]

I do several computations like this

bestValue = inf
parfor i = iIterator

    for j = 1:50
      thing = computeSomething(C(:,1:i),k(i+j))
      value = evalThing(thing,M);
      if value < bestValue
           bestValue = value;
           best_i_Idx = i;
      end 

end 

Now if M is large I get memory problems, since its copied to each local worker. However M is not changed.

Question :

1) How can I avoid local copies of M ?

2) Which is the best parallel structure for such an algorithm ?

3) I read about memmapfile. Is it possible to use it for that purpose ? So lets say, I save M to a file

 mValues.dat

Then I could pass each worker the object

m=memmapfile('mValues.dat','Format',{'double', [18000 18000],'x'})

But once I access it via

value = evalThing(thing,m.Data.x);

isn't that the same as giving each worker a copy in memory ? So does the worker produce a local copy of the matrix ?

Upvotes: 0

Views: 116

Answers (0)

Related Questions