MATLAB reading to the end of a binary file

Question

I think the solution will be quite simple for somebody with some MATLAB knowhow however I do not know how to do it.

I have a binary file that I am reading with fread and I am reading the first 4 bytes of this file followed by the next 2 bytes.

I basically want this process of reading 4 bytes followed by 2 bytes repeated till the end of the file is reached.

So the number of bytes read is 4,2,4,2,4,2......

I have the following to read the first pair of data and I want this to repeat.

fileID = fopen('MyBinaryFile');
4bytes = fread(fileID, 4);
fseek(fileID, 4, 0);
2bytes = fread(fileID, 2);

Thanks in advance for any help and suggestions

Answer 1

If you want to get the data without loops, there are MATLABish ways to that:

    %'Sizes'
    T = 4;    %'Time record size'
    D = 2;    %'Date record size'
    R = T+D;  %'Record size'

    %'Open file'
    f = fopen('MyBinaryFile', 'rb');
    if f < 0
            error('Could not open file.');
    end;

    %'Read the entire file at once, and close file'
    buf = fread(f, Inf, '*uint8');
    fclose(f);

    %'Ignore the last unpadded bytes, and reshape by the size of 1 record'
    buf = reshape(buf(1:R*fix(numel(buf)/R)), R, []);

    %'Pinpoint the data'
    time_bytes = buf(  1:  T, :);
    date_bytes = buf(T+1:T+D, :);

Answer 2

fileID = fopen('MyBinaryFile');
kk=1;
while ~feof(fileID)
   bytes4(kk) = fread(fileID, 4);
   fseek(fileID, 4, 0);
   bytes2(kk) = fread(fileID, 2);
   kk=kk+1;
end

the while loop condition is ~feof, which stands for End-Of-File. So as long as you haven't reached the end of your file it runs.

I added the kk just so you store everything and not just overwrite them each loop iteration.

Answer 3

I take it this is a variant of your former question MATLAB reading a mixed data type binary file.

Your goal is to read a binary file containing mixed data type. In your case it contains 2 columns:
1x single value (4 bytes) and 1x int16 value (2 bytes).

There are several ways to read this type of file. They differ in speed because some ways minimize disk access but require more temporary memory, and other way use just the memory needed but require more disk access (= slower).

Ultimately, the 3 ways I'm going to show you produce exactly the same result.

The direct answer to this question is the version #3 below, but I encourage you to have a look at the 2 other options described here, they are both really worth understanding.

---

For the purpose of the example, I had to create a binary file as you described. This is done this way:

%% // write example file
A = single(linspace(-3,1,11)) ;  %// a few "float" (=single) data
B = int16(-5:5) ;                %// a few "int16" data

fileID = fopen('testmixeddata.bin','w');
for il=1:11
    fwrite(fileID,A(il),'single');   
    fwrite(fileID,B(il),'int16');   
end
fclose(fileID);

This create a 2 column binary file, the columns being:

• 11 values of type float going from -3 to 1.
• 11 values of type int16 going from -5 to +5.

For future reference:

>> disp(A)
   -3.0000   -2.6000   -2.2000   -1.8000   -1.4000   -1.0000   -0.6000   -0.2000    0.2000    0.6000    1.0000
>> disp(B)
     -5     -4     -3     -2     -1      0      1      2      3      4      5

In each of the solution below, the first column will be read in a variable called varSingle, and the second column in a variable called varInt16.

---

1) Read all data in one go - convert to proper type after

%% // SOLUTION 1 (fastest) : Read all data in one go - convert to proper type after
fileID = fopen('testmixeddata.bin');
R = fread(fileID,'uint8=>uint8') ;       %// read all values, most basic data type (unsigned 8 bit integer)
fclose(fileID);

colSize = [4 2] ; %// number of byte for each column [4 byte single, 2 byte int16]

R = reshape( R , sum(colSize) , [] ) ;          %// reshape data into a matrix (6 is because 4+2byte=6 byte per column)
temp = R(1:4,:) ;                               %// extract data for first column into temporary variable (OPTIONAL)
varSingle = typecast( temp(:) , 'single' ) ;    %// convert into "single/float"
temp = R(5:end,:) ;                             %// extract data for second column
varInt16  = typecast( temp(:) , 'int16' ) ;     %// convert into "int16"

This is my favourite method. Specially for speed because it minimizes the read/seek operations on disk, and most post calculations are done in memory (much much faster than disk operations).

Note that the temporary variable I used was only for clarity/verbose, you can avoid it altogether if you get your indexing into the raw data right.

The key thing to understand is the use of the typecast function. And the good news is it got even faster since 2014b.

---

2) Read column by column (using "skipvalue") - 2 pass approach

%% // SOLUTION 2 : Read column by column (using "skipvalue") - 2 pass approach
col1size = 4 ;                                  %// size of data in column 1 (in [byte])
col2size = 2 ;                                  %// size of data in column 2 (in [byte])

fileID    = fopen('testmixeddata.bin');
varSingle = fread(fileID,'*single',col2size) ;  %// read all "float" values, skipping all "int16" 
fseek(fileID,col1size,'bof') ;                  %// rewind to beginning of column 2 at the top of the file
varInt16  = fread(fileID,'*int16',col1size) ;   %// read all "int16" values, skipping all "float" 
fclose(fileID);

That works too. It works fine ... but it is going to be slower than method 1 above, because you will have to scan the file twice. It may be a good option if the file is very large and method 1 above fail because of an out of memory error.

---

3) Read element by element

%% // SOLUTION 3 : Read element by element (slow - not recommended)
fileID = fopen('testmixeddata.bin');
varSingle=[];varInt16=[];
while ~feof(fileID)
    try
        varSingle(end+1) = fread(fileID, 1, '*single' ) ;
        varInt16(end+1)  = fread(fileID, 1, '*int16'  ) ;
    catch
        disp('reached End Of File')
    end
end
fclose(fileID);

That does work too, and if you were writing C code it would be more than ok. But in Matlab this is not the recommended way to go (your choice ultimately)

---

As promised, the 3 methods above will give you exactly what we wrote in the file at the beginning:

>> disp(varSingle)
   -3.0000   -2.6000   -2.2000   -1.8000   -1.4000   -1.0000   -0.6000   -0.2000    0.2000    0.6000    1.0000
>> disp(varInt16)
    -5    -4    -3    -2    -1     0     1     2     3     4     5