CODEWITHSUNDEEP
pythontcpstruct
petebachant
petebachant

Reputation: 87

Unpacking data structure with Python

I am receiving some data from a device over a TCP socket. I have the specifications for the structure, but don't know how to use this to unpack it. I'm assuming this means writing a formatting string for the struct.unpack function, but I am having trouble figuring that out given the specs from the manufacturer. Each of these 32-bit pieces is a "DWORD", but I'm not sure how to interpret that, and pull out the relevant bits:

The status header is a 22x32bit (88 bytes) structure of mixed data types that precedes and provides information about the data being transferred. The following table shows the relevant members of the structure. The shaded member fields are reserved for future expansion, used internally for instrument monitoring and controls, or simply not yet implemented.

Imgur Imgur

Here are two sample structures:

b'\x08\x04i!\x13\x02\x00\x1f\x00\x00\x80\x0c\x01\x00p\x00\x00\x00\x00\x00\x00\x00\x00\x00\x84\x9e\x00\x00\x85\x9e\x01\x00:B\x04\x00\x08\xf4DT\x01\x00k\x01\xdc\x8c\x00\x00c\x03X\x00\x9eR\xa4QTV\xf0U\xd0\x83\xd0\x83\xd0\x83\xd0\x83\x01\x00\x00\x00@B\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

b'\x08Ti!\x1a\x02\x00\x1f\x00\x00x\x0c\x01\x00p\x00\x00\x00\x00\x00\x00\x00\x00\x00\xd4\x91\x00\x00\xd5\x91\x01\x00`y\x00\x00\x05\xf4DT\x01\x00\xbb\x00\xdb\x8c\x00\x00c\x03X\x00\x9eR\xa4QTV\xf0U\xd0\x83\xd0\x83\xd0\x83\xd0\x83\x01\x00\x00\x00@B\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'

Upvotes: 1

Views: 1359

Answers (1)

davidism
davidism

Reputation: 127260

This should get you started. Unpack the received bytes using struct.unpack, then do some parsing for fields that are represented by less than 1 byte of data.

from struct import unpack

# data recieved, 88 random bytes for example purpose
data = b'\xdb[\x91wdI\t\xef\xc6c\xde\x14\xac\x1e\x08\x10.f\xc0\xbd\xfd\xa15\x8cP\x101\xed\xc5\xd9\x98X\xb5\xc2\x00Z\xd2\xb9\xb0Xa\x04\xfa\xb8\xceA\x94_7\xc7\xde\t\xf2kX\x9d2\xc3\x84\xb3\x19\x8e\xf5\x99\xc3\xba\x08\xaa0$\x17\xfbd\xbb\x7f\xfd&\xf5\x1aU\t`\x11@zD\xce\xff'

# unpack the struct into variables
(
    abcde, fw_ver, cur_layer, fs_radix,  # 0 needs parse
    fghij, knpl, fbg_thermistor,  # 1 needs parse
    tx_ambient_temp, reserved2,  # 2
    num_ffpi_peaks, num_fbg_peaks,  # 3
    num_dut2_peaks, num_dut1_peaks,  # 4
    num_dut4_peaks, num_dut3_peaks,  # 5
    reserved7, qr, acq_counter3,  # 6 needs parse
    serial_number,  # 7
    kernel_timestamp_microseconds,  # 8
    kernel_timestamp_seconds,  # 9
    kernel_src_buffer, kernel_buffers,  # 10
    error_and_kernel_rt_loc0,  # 11 needs parse
    header_length, header_ver, buffers,  # 12
    dut2_gain, dut1_gain,  # 13
    dut4_gain, dut3_gain,  # 14
    dut2_noise_thresh, dut1_noise_thresh,  # 15
    dut4_noise_thresh, dut3_noise_thresh,  # 16
    hw_clk_div, peak_data_rate_div,  # 17
    granularity,  # 18
    reserved4,  # 19
    starting_lambda,  # 20
    ending_lambda  # 21
) = unpack(
    '>'  # big endian
    'BBBB'  # 0 needs parse
    'BBH'  # 1 needs parse
    'HH'  # 2
    'HH'  # 3
    'HH'  # 4
    'HH'  # 5
    'BBH'  # 6 needs parse
    'I'  # 7
    'I'  # 8
    'I'  # 9
    'HH'  # 10
    'I'  # 11 needs parse
    'HBB'  # 12
    'HH'  # 13
    'HH'  # 14
    'HH'  # 15
    'HH'  # 16
    'HH'  # 17
    'I'  # 18
    'I'  # 19
    'I'  # 20
    'I',  # 21
    data
)

# 0 parse abcde
acq_triggered = bool(abcde & 0x80)
calibration_fault = bool(abcde & 0x40)
start_of_frame = bool(abcde & 0x20)
primary_fan_state = bool(abcde & 0x10)
secondary_fan_state = bool(abcde & 0x08)
s0_mux_state = bool(abcde & 0x04)
s1_mux_state = bool(abcde & 0x02)
s2_mux_state = bool(abcde & 0x01)

# 1 parse fghij
xfer_type = fghij >> 4
soa_therm_limit = bool(fghij & 0x08)
soa_current_limit = bool(fghij & 0x04)
tec_over_temp = bool(fghij & 0x02)
tec_under_temp = bool(fghij & 0x01)

# 1 parse knpl
operating_mode = knpl >> 6
triggering_mode = (knpl & 0x30) >> 4
sm041_mux_level = (knpl & 0x0c) >> 2
sw_position = knpl & 0x03

# 6 parse qr
nrz_command = qr >> 5
reserved6 = qr & 0x1f

# 11 parse
error = error_and_kernel_rt_loc0 >> 24
kernel_rt_loc0 = error_and_kernel_rt_loc0 & 0xffffff

I assumed big endian since this is coming over TCP, but that might be wrong. If all the data looks off, try using < for little endian. Or if you're unlucky some values might be one while some are the other, in which case you'll have to split this into multiple unpacks. You'll also have to further process these values, as I'm sure some of them aren't supposed to be interpreted as ints.

The unpack format can be shortened to

(all, those, variables) = unpack('>6B9H2BH3I2HIH2B10H4I', data)

but I don't think it's as clear.

Upvotes: 2

Related Questions