CODEWITHSUNDEEP
pythonarraysdata-conversion
Jay Leno
Jay Leno

Reputation: 11

Converting string which contains bytearray to uint16

I have a large bytearray which I store to a file, for post-processing purposes. For this example, let's assume the data is a counter from 0 to 3 base 10.

The format I retrieve the data in from a certain method is a bytearray, and I need to store it efficiently as the simulation requires a large data-set. Therefore I store it as 

f=open('data_0.txt','w')
f.write(str(b_a))
f.close()

when I read it I get a string of data: data_read = b'\x00\x00\x01\x00\x02\x00\x03\x00'

f=open('data_0.txt')
data_read = f.read()
f.close()

I expect a function to take the string: b'\x00\x00\x01\x00\x02\x00\x03\x00' and convert it to an array of [0, 1, 2, 3].

When I use struct.unpack("BBBBBBBB" , data_read) I would expect 0,0,1,0,2,0,3,0, instead it just displays the ascii for b,',\,x, etc., it is the same for ord(0), ord(1) etc.

Is there any convenient function or pythonic manner to go from a string: b'\x00\x00\x01\x00\x02\x00\x03\x00' and convert it to an array of [0, 1, 2, 3]?

Upvotes: 1

Views: 253

Answers (2)

Serge Ballesta
Serge Ballesta

Reputation: 148975

If you get the ascii display for b,',\,x, etc., it means that the file actually contains it. In fact it is caused by the str(b_a) call.

Demo:

>>> b_a = b'\x00\x00\x01\x00\x02\x00\x03\x00'
>>> s_a = str(b_a)
>>> print(s_a)
b'\x00\x00\x01\x00\x02\x00\x03\x00'
>>> [hex(ord(x)) for x in s_a]
['0x62', '0x27', '0x5c', '0x78', '0x30', '0x30', '0x5c', '0x78', '0x30', '0x30', '0x5c', '0x78', '0x30', '0x31', '0x5c', '0x78', '0x30', '0x30', '0x5c', '0x78', '0x30', '0x32', '0x5c', '0x78', '0x30', '0x30', '0x5c', '0x78', '0x30', '0x33', '0x5c', '0x78', '0x30', '0x30', '0x27']

which is the display for b, ', \, x, 0, etc.

ast.literal_eval can be used to convert it back to a byte string. Demo:

>>> b2 = ast.literal_eval(s_a)
>>> b2 == b_a
True
>>> struct.unpack('HHHH', b2)
(0, 1, 2, 3)

So if you want to keep the str call when writing, you have to read it that way:

with open('data_0.txt') as f:
    data_read = f.read()
data = struct.unpack('HHHH', ast.literal_eval(data_read))

Upvotes: 0

han solo
han solo

Reputation: 6590

You need to use shorts or unsigned shorts:)

>>> import struct
>>> data_read
b'\x00\x00\x01\x00\x02\x00\x03\x00'
>>> struct.unpack("hhhh" , data_read)
(0, 1, 2, 3)
>>> struct.unpack("HHHH" , data_read)
(0, 1, 2, 3)

Upvotes: 1

Related Questions