CODEWITHSUNDEEP
pythonstructbyteendianness
Thomas Johnson
Thomas Johnson

Reputation: 11688

Why does python's struct think little-endian and big-endian imply different lengths?

I thought that little-endian and big-endian numbers were the same size (in bytes). But python 2.7's struct module says this:

In [46]: struct.unpack('>L', datalen[4:8])[0]
Out[46]: 35098131

In [47]: struct.unpack('L', datalen[4:8])[0]
---------------------------------------------------------------------------
error                                     Traceback (most recent call last)
<ipython-input-47-f18e2a303d6c> in <module>()
----> 1 struct.unpack('L', datalen[4:8])[0]

error: unpack requires a string argument of length 8

Why is the big endian long 4 bytes but struct expects the little-endian long to be 8 bytes?

Upvotes: 1

Views: 2391

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1122372

The default is @ or native order, which is not necessarily little endian.

Native order uses native alignment:

By default, C types are represented in the machine’s native format and byte order, and properly aligned by skipping pad bytes if necessary (according to the rules used by the C compiler).

(emphasis mine) and

Native byte order is big-endian or little-endian, depending on the host system. For example, Intel x86 and AMD64 (x86-64) are little-endian; Motorola 68000 and PowerPC G5 are big-endian; ARM and Intel Itanium feature switchable endianness (bi-endian). Use sys.byteorder to check the endianness of your system.

It is the alignment that causes the size to differ, not the endianess.

The C data structure alignment is used to improve memory performance; you need to make sure you pick the right type for your data input. The C native alignment for a long is:

A long long (eight bytes) will be 8-byte aligned.

To compare between little and big endianess without native alignment, use < and > when comparing:

struct.unpack('<L', datalen[4:8])[0]

Upvotes: 3

Wooble
Wooble

Reputation: 89927

Specifying 'L' without > or < is not "little-endian"; it's native endianness and native size. The standard size is 4 bytes, but your machine's native size is 8.

If you explicitly want standard sized little-endian, use:

struct.unpack('<L', datalen[4:8])[0]

Upvotes: 7

Related Questions