CODEWITHSUNDEEP
optimizationcsvconvertersoctave
Federico Capello
Federico Capello

Reputation: 1118

Save CSV to .MAT or binary

I'm working on Octave and have a ~80 mb .csv file to load.

csvread() takes a long time to load.

How can I save a csv file to a .mat file or binary or, at least, speed up Octave csvread()?

Thank you in advance!

Upvotes: 2

Views: 6947

Answers (3)

user3207297
user3207297

Reputation: 53

I have faced this issue a couple times. What I do might be pretty sloppy, but it works well for me. I open the file in a spreadsheet (Excel or Calc), copy the contents without any header and paste in a text editor, from which I save as a .mat

(Gedit and Notepad++ are my editors of choice when dealing with large files, by the way)

Upvotes: 0

brm
brm

Reputation: 3806

This is a python version (using numpy and scipy). The program takes two arguments, the input csv file (assumed to be all numerical data, no header strings for example) and the output filename of the mat file. When loading the mat file, the data in it is accessible through the matlab variable name csvmatrix.

import csv  
import sys
import numpy
import scipy.io

if __name__ == "__main__":
    if len(sys.argv) != 3:
        print "Usage: %s input.csv output.mat" % sys.argv[0]
        sys.exit(-1)

    data = [ ]
    with open(sys.argv[1]) as f:
        reader = csv.reader(f)
        for row in reader:
            rowData = [ float(elem) for elem in row ]
            data.append(rowData)

    matrix = numpy.array(data)
    scipy.io.savemat(sys.argv[2], {'csvmatrix':matrix})

Upvotes: 1

LPH
LPH

Reputation: 1295

well, if you are also using r, you could do the following:

  1. use "read.csv" in R to read in the file;
  2. use the R package"R.matlab" to write a ".mat" file

now you can load the .mat file "faster" in Octave- but step 1 ("read csv") can also be pretty slow;

Upvotes: 2

Related Questions