CODEWITHSUNDEEP
pythonpandasnumpyrecarray
Daniel Lima
Daniel Lima

Reputation: 995

Numpy recarray from bytes or string

I have a process where I need to convert a numpy recarray to bytes, and after that reconstruct the recarray from the bytes.

However, I am not sure how to do recover the array from bytes.

Does anyone know how could I do it?

Example code:

import numpy as np
import pandas as pd

df = pd.DataFrame(np.zeros(500))
rec = df.to_records()
rec_s = rec.tostring()  # this returns a bytes object

# perform some computation

new_rec = <method to recover from bytes>(rec_s)

Note: I don't actually need to use numpy recarry, just some structure that will allow me to transform the pandas dataframe into a bytes object, and also recover it.

Upvotes: 0

Views: 414

Answers (1)

hpaulj
hpaulj

Reputation: 231335

In [497]: arr = np.ones(3, dtype='i,i,f')                                                      
In [498]: arr                                                                                  
Out[498]: 
array([(1, 1, 1.), (1, 1, 1.), (1, 1, 1.)],
      dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<f4')])
In [499]: astr = arr.tostring()                                                                
In [500]: astr                                                                                 
Out[500]: b'\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80?\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80?\x01\x00\x00\x00\x01\x00\x00\x00\x00\x00\x80?'

Recover it using the same dtype:

In [502]: np.frombuffer(astr, arr.dtype)                                                       
Out[502]: 
array([(1, 1, 1.), (1, 1, 1.), (1, 1, 1.)],
      dtype=[('f0', '<i4'), ('f1', '<i4'), ('f2', '<f4')])

If the source was 2d, you'd have to reshape as well

Upvotes: 1

Related Questions