CODEWITHSUNDEEP
matlaboptimizationmatrixsaveoctave
Ovidiu
Ovidiu

Reputation: 75

Save/load a large matrix in Octave

I'm playing with large pointcloud data in Octave (different files ranging from [10^5 to 10^7, 4] elements) and I'm looking for ways to optimize the code.

Right now I am trying to save the data into a .mat file as I've read somewhere (confirmation needed) that loading from a .mat file is much faster than loading the actual data.txt file every time.

save -ascii myfile data works fine needs since it's only numerical values I want to store but

load('myfile.mat') brings up a 1x1 matrix containing all the values instead of having a nx4 matrix, which is strange because when I use load('data.txt') I get a full nx4 matrix.

The problem seems to be with the save syntax. Any way I can save the file so I can load it with its original dimensions? Or do I have to manipulate the resulting 1x1 variable somehow?

Bonus question:
Browsing through some answers I kinda got the feeling that working with the transpose matrix instead of the nx4 would improve runtime considerably. Is that true?

Upvotes: 2

Views: 6884

Answers (1)

Andy
Andy

Reputation: 8091

Use a binary format if speed matters. Below a little speed comparison

a = rand (1e6, 4);
fn = tmpnam;

tic; save ("-ascii", fn, "a"); toc;
tic; load ("-ascii", fn); toc;
stat (fn).size

tic; save ("-v7", fn, "a"); toc;
tic; load ("-v7", fn); toc;
stat (fn).size

tic; save ("-v6", fn, "a"); toc;
tic; load ("-v6", fn); toc;
stat (fn).size

tic; save ("-binary", fn, "a"); toc;
tic; load ("-binary", fn); toc;
stat (fn).size

which gives

Elapsed time is 2.82237 seconds.
Elapsed time is 6.28686 seconds.
ans =  61000000
Elapsed time is 1.54074 seconds.
Elapsed time is 0.252718 seconds.
ans =  30192558
Elapsed time is 0.030833 seconds.
Elapsed time is 0.047183 seconds.
ans =  32000184
Elapsed time is 0.116342 seconds.
Elapsed time is 0.0523431 seconds.
ans =  32000045

As you can see -v6 is much faster than -ascii

EDIT: also keep in mind that "-ascii" only uses single precision floats

Upvotes: 5

Related Questions