CODEWITHSUNDEEP
pythonstruct
Alf_InPogform
Alf_InPogform

Reputation: 53

Unspecified byte lengths in Python

I'm writing a client for a P2P application at the minute and the spec for the protocol says that the header for each packet should have each field with a particular byte length like so:

Version: 1 Byte 
Type: 1 Byte
Length: 2 Bytes
And then the data

I've got the way of packing and unpacking the header fields (I think) like this:

packed = struct.pack('cch' , '1' , '1' , 26)

This constructs a header for a packet with a data length of 26, but when it comes to unpacking the data I'm unsure how to go about getting the rest of the data afterwards. To unpack we need to know the size of all the fields, unless I'm missing something? I guess to pack the data I'd use a format indicator 'cch26s' meaning:

1 Byte char
1 Byte char
2 Byte short
26 Byte char array

But how do I unpack the data when I don't know how much data will be included in the packet first?

Upvotes: 4

Views: 5353

Answers (2)

alexis
alexis

Reputation: 50220

The way you're describing the protocol, you should unpack the first four bytes first, and extract Length (a 16-bit int). This tells you how many bytes to unpack in a second step.

version, type, length = struct.unpack("cch", packed[:4])
content, = struct.unpack("%ds" % length, packed[4:])

This is if everything checks out. unpack() requires that the packed buffer contain exactly as much data as you unpack. Also, check whether the 4 header bytes are included in the length count.

Upvotes: 3

unutbu
unutbu

Reputation: 880777

You can surmise the number of characters to unpack by inspecting len(data).

Here is a helper function which does this for you:

def unpack(fmt, astr):
    """
    Return struct.unpack(fmt, astr) with the optional single * in fmt replaced with
    the appropriate number, given the length of astr.
    """
    # http://stackoverflow.com/a/7867892/190597
    try:
        return struct.unpack(fmt, astr)
    except struct.error:
        flen = struct.calcsize(fmt.replace('*', ''))
        alen = len(astr)
        idx = fmt.find('*')
        before_char = fmt[idx-1]
        n = (alen-flen)/struct.calcsize(before_char)+1
        fmt = ''.join((fmt[:idx-1], str(n), before_char, fmt[idx+1:]))
        return struct.unpack(fmt, astr)

You can use it like this:

unpack('cchs*', data)

Upvotes: 2

Related Questions