CODEWITHSUNDEEP
matlabinterop
Midhat
Midhat

Reputation: 17810

How can I save a very large MATLAB sparse matrix to a text file?

I have a 30000x14000 sparse matrix in MATLAB (version 7), which I need to use in another program. Calling save won't write this as ASCII (not supported). Calling full() on this monster results in an Out of Memory error.
How do I export it?

Upvotes: 17

Views: 30145

Answers (8)

Code42
Code42

Reputation: 2482

Use this script: msm_to_mm.m, writes an MATLAB sparse matrix to an MatrixMarket file.

And This thread may also be useful.

Upvotes: 0

Matthieu
Matthieu

Reputation: 626

You can use find to get index & value vectors:

[i,j,val] = find(data)
data_dump = [i,j,val]

You can recreate data from data_dump with spconvert, which is meant to "Import from sparse matrix external format" (so I guess it's a good export format):

data = spconvert( data_dump )

You can save to ascii with:

save -ascii data.txt data_dump

But this dumps indices as double, you can write it out more nicely with fopen/fprintf/fclose:

fid = fopen('data.txt','w')
fprintf( fid,'%d %d %f\n', transpose(data_dump) )
fclose(fid)

Hope this helps.

Upvotes: 29

Midhat
Midhat

Reputation: 17810

I saved it as text using Java within MATLAB. MATLAB Code:


pw=java.io.PrintWriter(java.io.FileWriter('c:\\retail.txt'));
line=num2str(0:size(data,2)-1);
pw.println(line);
for index=1:length(data)
    disp(index);
    line=num2str(full(data(index,:)));
    pw.println(line);
end
pw.flush();
pw.close();

Here data is an extremely large sparse matrix.

Upvotes: 3

ehsan
ehsan

Reputation: 11

dlmwrite - Write matrix to ASCII-delimited file Syntax

dlmwrite(filename, M)

dlmwrite(filename, M, 'D')

dlmwrite(filename, M, 'D', R, C)

dlmwrite(filename, M, 'attrib1', value1, 'attrib2', value2, ...)

dlmwrite(filename, M, '-append')

dlmwrite(filename, M, '-append', attribute-value list)

Upvotes: 1

Mr Fooz
Mr Fooz

Reputation: 111856

Use the find function to get the indices of non-zero elements...

idcs = find(data);
vals = data(idcs);
...save the index vector and value vector in whatever format you want...

If you want, you can use ind2sub to convert the linear indices to row, column subscripts.

If you need to recreate a sparse matrix in matlab from subscripts + values, use spconvert.

Upvotes: 2

Scottie T
Scottie T

Reputation: 12195

If this is pretty much a one time deal, then I would just iterate through the matrix and write the matrix to an ASCII file by brute force, or else use @Veynom's suggestion and call full() on a subset of rows. It may take a while, but it will probably be done faster than it might take to learn how to read in a .mat file outside of the MATLAB environment.

If this is something you need to do on a recurring basis, then I would take @Vebjorn's advice and use a library to read the .mat file.

Upvotes: 0

Vebjorn Ljosa
Vebjorn Ljosa

Reputation: 18008

Save the sparse matrix as a .mat file. Then, in the other program, use a suitable library to read the .mat file.

For instance, if the other program is written in Python, you can use the scipy.io.mio.loadmat function, which supports sparse arrays and gives you a sparse numpy matrix.

Upvotes: 8

Veynom
Veynom

Reputation: 4147

Did you try partitioning it ?

I mean try calling full() on the 1000 first rows (or 5000) and then repeat the process if it works.

Upvotes: 2

Related Questions