CODEWITHSUNDEEP
matlabparsingcsvcommentsoctave
John Lawrence Aspden
John Lawrence Aspden

Reputation: 17470

Can I ignore comment lines while reading in a csv file in Octave/MATLAB?

I've got a data file that looks like:

# data file
# blah
# blah

       0.000000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN
       0.020000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN
       0.040000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN
       0.060000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN
       0.080000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN
       0.100000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN
       0.120000, 0.0, 24.198, 6.864,NaN,NaN,NaN,NaN

and I'd like to read it with an Octave program.

csvread(file,3,0) works perfectly in this case, but I worry about having to work out the 3 by hand.

Is there some way to say 'throw away any lines starting with #, and any blank lines, before doing the csvread'?

Upvotes: 3

Views: 4912

Answers (3)

Andy
Andy

Reputation: 8091

In octave you could do

d = load("yourfile")

which should ignore the # lines

Edit: The above uses autodetection of the file type, you could also force it with d = load ("-ascii", "yourfile"). Quote from help load:

 '-ascii'
      Force Octave to assume the file contains columns of numbers in
      text format without any header or other information.  Data in
      the file will be loaded as a single numeric matrix with the
      name of the variable derived from the name of the file.

Unfortunately the help doesn't mention that lines starting with % or # are ignored. For this you have to look at the source code (which is fortunately available since GNU Octave is free software) get_mat_data_input_line from octave source

From there you can see that all chars after % or # are skipped.

Upvotes: 4

ederag
ederag

Reputation: 2509

Here is a way to skip header lines starting with a comment string. The csvread line could be replaced by a dlmread call for delimiters other than ','. Both these functions are much faster than textscan on octave 3.8.2.

fid = fopen('csvFile.csv','r');

comment = '#';
while strcmp(fgets(fid, length(comment)), comment)
    % line begins with a comment, skip it
    fskipl(fid);
endwhile
% get back, because the last read length(comment) characters
% are not comments, actually
fseek(fid, -length(comment), SEEK_CUR);

c = csvread(fid);

fclose(fid);

Upvotes: 1

Jonas
Jonas

Reputation: 74940

csvread does not allow this option. Instead, you can use textscan, but then, you need to know how many columns (or rows) your csv file has.

For example:

fid = fopen('csvFile.csv','r');
c = textscan(fid,'%f','commentStyle','#','delimiter',',');
fclose(fid); %# close the file as soon as we don't need it anymore

array = reshape([c{:}],[],7)';

Upvotes: 3

Related Questions