CODEWITHSUNDEEP
pythonarrayspython-3.x
kittygirl
kittygirl

Reputation: 2443

How to read huge binary file and get a certain columns array in Python3?

I try to get python array from a soapy binary file.this binary file size is 6G,has 12 columns with uncertain rows, looks like below:

ss2017-03-17, 13:18:25, 88000000.0, 90560000.0, 426666.666667, 647168, -98.6323, -98.7576, -97.3716, -98.3133, -98.8829, -98.9333
ss2017-03-17, 13:18:25, 90560000.0, 93120000.0, 426666.666667, 647168, -95.7163, -96.2564, -97.01, -98.1281, -90.701, -88.0872
ss2017-03-17, 13:18:25, 93120000.0, 95680000.0, 426666.666667, 647168, -99.0242, -91.3061, -91.9134, -85.4561, -86.0053, -97.8411
ss2017-03-17, 13:18:26, 95680000.0, 98240000.0, 426666.666667, 647168, -94.2324, -83.7932, -78.3108, -82.033, -89.1212, -97.4499

After

f = np.fromfile(open('filename','rb'))
print(f.ndim)

I got a one dimension array.
How to read this binary file and get array has 12 elements per row?

Upvotes: 0

Views: 262

Answers (2)

pletnes
pletnes

Reputation: 479

I'd just reshape it with -1, this will infer the number of rows.

f = np.fromfile(open('filename','rb'))
f = f.reshape(-1, 12)
print(f.ndim)

Note that this will fail if the size of the array is not a multiple of 12, for whatever reason. I'd suggest checking the shape first if this matters to you.

Upvotes: 0

Rahul
Rahul

Reputation: 11560

You can reshape your data like this:

np.array(data).reshape(-1)

Upvotes: 1

Related Questions