CODEWITHSUNDEEP
pythonstructpack
Kris
Kris

Reputation: 97

struct pack printing a string instead of binary value

Why is my struct.pack printing a string instead of a binary stream?

var =  struct.pack('hhl3sf', 1, 2, 3, 'm6y', 2.7)
print repr(var)

The output is:

'\x01\x00\x02\x00\x03\x00\x00\x00m6y\x00\xcd\xcc,@'

Should the 'm6y' be printed as \x6d\x36\x79? And if not, how can I print it directly from pack as her or just plain binary? And why is the char @ print at the end? Thanks.

I tried converting all value to ascii and then print them in hex. And also running binascii.hexlify. Both of them works, but I'm wondering why pack is not doing this automatically. Thanks.

map(lambda c: ord(c), var)
map(lambda i: hex(i), map(lambda c: ord(c), var))


print 'Packed value : ', binascii.hexlify(var)

Upvotes: 2

Views: 1420

Answers (2)

CristiFati
CristiFati

Reputation: 41116

String printing question

In [SO]: Python struct.pack() behavior (@CristiFati's answer) (somewhere at the beginning), I try to briefly explain the behavior. Basically, when displaying some memory contents to the user, if a certain byte has a corresponding printable character, that one is used (note that this is for presentation only, it doesn't affect the memory contents).
If you want to bypass this behavior, you'd have to do it manually (there are a number of ways - I'm not going to insist on them). But bear in mind that by doing so, the resulting string will not be equivalent to the original one:

>>> b0 = b"\x41"
>>> b0
b'A'
>>> len(b0)
1
>>>
>>> b1 = "".join(("\\x{:02X}".format(c) for c in b0))
>>> b1
'\\x41'
>>> len(b1)
4

Ending char question

Posting [Python 3.Docs]: struct - Interpret bytes as packed binary data page.

Breaking the problem in simpler ones and tackle each of them separately:

  • The '@' char (0x40) at the end of the resulting string: it's part of 2.7's representation:

    >>> import struct
>>>
>>> s0 = struct.pack(b"f", 2.7)
>>> s0
b'\xcd\xcc,@'
>>> len(s0)
4
>>>
>>> s1 = struct.pack(b"f", 1.7)
>>> s1
b'\x9a\x99\xd9?'
>>> len(s1)
4

  • I believe that the confusion was generated by the last '\x00' byte (var[-5]): 

    >>> struct.pack(b"3sI", b"ABC", 0xFFFFFFFF)
b'ABC\x00\xff\xff\xff\xff'
>>>
>>> struct.pack(b"@3sI", b"ABC", 0xFFFFFFFF)
b'ABC\x00\xff\xff\xff\xff'
>>>
>>> struct.pack(b"=3sI", b"ABC", 0xFFFFFFFF)
b'ABC\xff\xff\xff\xff'
>>>
>>>
>>> struct.pack(b"4sI", b"ABCD", 0xFFFFFFFF)
b'ABCD\xff\xff\xff\xff'
>>>
>>> struct.pack(b"1sI", b"A", 0xFFFFFFFF)
b'A\x00\x00\x00\xff\xff\xff\xff'
>>>
>>> struct.pack(b"1sH", b"A", 0xFFFF)
b'A\x00\xff\xff'
>>>
>>> struct.pack(b"1sB", b"A", 0xFF)
b'A\xff'
>>>
>>> struct.pack(b"1sd", b"A", 2.7)
b'A\x00\x00\x00\x00\x00\x00\x00\x9a\x99\x99\x99\x99\x99\x05@'

    As seen from the above examples, it doesn't have anything to do with our float number, but with the string before, and it's a matter of alignment (which for float is 4 bytes). Check [Wikipedia]: Data structure alignment for more details

Upvotes: 0

BoarGules
BoarGules

Reputation: 16952

Python's default representation prints bytestrings as ascii characters where this is possible and only as \x escapes if not.

It is often expected that 'm6y' be printed as \x6d\x36\x79 but that is not how it works.

If you want to see the variable displayed all as hex then you need to do that yourself. For example:

>>> "".join("{0:02x}".format(b) for b in var)
'01000200030000006d367900cdcc2c40'
>>> print("".join("\\0x{0:02x}".format(b) for b in var))
\0x01\0x00\0x02\0x00\0x03\0x00\0x00\0x00\0x6d\0x36\0x79\0x00\0xcd\0xcc\0x2c\0x40

Exactly how you want to see it is up to you. Bytestrings are often ascii characters, and displaying b"Hello world" by default as 

\0x48\0x65\0x6c\0x6c\0x6f\0x20\0x77\0x6f\0x72\0x6c\0x64

isn't friendly.

Upvotes: 1

Related Questions