CODEWITHSUNDEEP
matlab
user2521074
user2521074

Reputation: 61

Matlab euclidean pairwise square distance function

I have a 60000 by 300 matrix call X. I am trying to find pairwise euclidean distances. I know that the pdist function in matlab (stats toolbox) can do this. However, when I type in the code pdist(X), I get the following error message:

Error using pdistmex
Out of memory. Type HELP MEMORY for your options.

Error in pdist (line 252)
    Y = pdistmex(X',dist,additionalArg);

Any advice for fixes? Is the matrix size too big?

Upvotes: 0

Views: 1137

Answers (1)

voxeloctree
voxeloctree

Reputation: 849

Simply put yes, the pdist method is hungry for your memory and your computer cannot feed it. For example, even with a 6000 by 300 matrix X, I get the following variable sizes for X and Y using whos X Y:

>> whos X Y
  Name         Size                      Bytes  Class     Attributes

  X         6000x300                  14400000  double              
  Y            1x17997000            143976000  double

Now my my memory states (on a 32 bit machine):

>> memory
        Maximum possible array:             677 MB (7.101e+008 bytes) *

So I am really pushing the memory limits with the computation Y = pdist(X) as this produces an array of roughly 1.44 *10^8 bytes whereas the maximum possible array size is roughly of order 5 times that. Any bigger with the matrix and your system might not be happy. Your matrix of 60000 by 300 will produce a Y array of 179970000 values!

There might be workarounds if you really need to compute the Euclidian distance of a matrix this size, if so, I might be able to help you more...

Upvotes: 1

Related Questions