CODEWITHSUNDEEP
performancematlabfile-io
Mario Krenn
Mario Krenn

Reputation: 223

Matlab: Fast reading of file with many lines of (%d %d %ld)

I am kind of frustrated with fscanf and its time-performance in reading a file with structured data. I want to read a .txt file, which has three entries per line: DOUBLE DOUBLE LONG-DOUBLE, and I only want to read the first N entries. Unfortunatly, fscanf is very slow. Do you know any faster method?

Btw, I am aware of several topics on this topic here, for instance this question. However, the answer does not help in my case, as i'm already using fscanf.

My code is:

    formatSpec='%d %d %ld'; % important: last one is long-double to support 64bit values!
    sizeA=[3 100000];
    file1=fopen('file1.txt','r');
        [content,cc]=fscanf(file1,formatSpec,sizeA);        
    fclose(file1);

Do you know any more clever idea to read N lines of a file with the given structure? Thanks!

Edit: The filecontent of file1.txt looks like this:

1  1 204378259709308
0  1 204378259782523
1  1 204378260105693
3  1 204378260381676
3  1 204378260854931
1  1 204378261349990
1  1 204378262189528
0  1 204378263067715
1  1 204378263301204
1  1 204378263676471
1  1 204378263771064
1  1 204378264565420
0  1 204378264608240
0  1 204378264973698
...
3  1 205260543966542

So basicly: A[space][space]B[space]C with A and B are [0,9] and C is a 64bit integer

Upvotes: 1

Views: 304

Answers (2)

Divakar
Divakar

Reputation: 221614

You could use textscan here for reading first N entries, which is supposedly pretty fast in latest versions of MATLAB -

fid = fopen(inputfile);             %// inputfile is the path to the input text file  
C = textscan(fid, '%d %d %d64',N);  %// N is the number of first entries to be read
fclose(fid);

%// Get data into three separate variables (if needed)
col1 = C{:,1};
col2 = C{:,2};
col3 = C{:,3};

Upvotes: 2

arccoder
arccoder

Reputation: 57

% To read all the rows and columns
T = dlmread('file1.txt',' ');

% To read specific rows and columns
% R1 - First row to read 
% C1 - First column to read 
% R2 - First row to read 
% C2 - First column to read 
% First row or column index is 0

% Following is the code to read rows 3, 4 and 5 
T = dlmread('file1.txt',' ',[2 0 4 2]);

By default it will read as double.

To get integer values

A = uint8(T(:,1));
B = uint8(T(:,2));
C = uint64(T(:,3));

Hope this helps :)

Upvotes: 0

Related Questions