CODEWITHSUNDEEP
pythonbinaryfiles
Mephisto
Mephisto

Reputation: 690

How to read a binary file with a 32-bit float, a 64-bit float and another-32 bit float in Python

This is a binary file with a very simple structure for learning purposes. Each register has 3 numbers: a 32-bit float, a 64-bit float and another 32-bit float. If I dump it on the screen in hexadecimal, it looks like this:

0000000: 0800 0000 0000 0000 0000 0000 0800 0000  ................
0000010: 0800 0000 0000 0000 0000 f03f 0800 0000  ...........?....
0000020: 0800 0000 182d 4454 fb21 0940 0800 0000  .....-DT.!.@....

(...)

If I manually copy the third line in binary format, I can read it into three variables:

import struct
data = b'\x08\x00\x00\x00\x18-DT\xfb!\t@\x08\x00\x00\x00'
l1, value, l2 = struct.unpack("<idi", data)
# (8, 3.141592653589793, 8)

That works, but I need to read the file from disk, not only manually copying each register in binary, because I need to do this with millions data. I need something equivalent to the following command used in ascii files:

l1, value, l2 = pylab.loadtxt('./test_file.binary',unpack=True)

Which doesn't work here.

Upvotes: 2

Views: 3049

Answers (2)

Denis
Denis

Reputation: 97

You may try this method

# includes core parts of numpy, matplotlib
import matplotlib.pyplot as plt
import numpy as np
# include scipy's signal processing functions
import scipy.signal as signal


# practice reading in complex values stored in a file
# Read in data that has been stored as raw I/Q interleaved 32-bit float samples
dat = np.fromfile("iqsamples.float32", dtype="float32")
# Look at the data. Is it complex?


dat = dat[0::2] + 1j*dat[1::2]
print(type(dat))
print(dat)

# # Plot the spectogram of this data
plt.specgram(dat, NFFT=1024, Fs=1000000)
plt.title("PSD of 'signal' loaded from file")
plt.xlabel("Time")
plt.ylabel("Frequency")
plt.show()  # if you've done this right, you should see a fun surprise here!

Upvotes: 0

wim
wim

Reputation: 362945

Read the file in binary mode:

def read_stuff(fname='test_file.binary'):
    with open(fname, mode='rb') as f:
        while True:
            data = f.read(16)
            if len(data) < 16:
                # end of file
                return
            yield struct.unpack("<idi", data)

This is a generator. To consume it:

for l1, value, l2 in read_stuff():
    ...

Upvotes: 3

Related Questions