CODEWITHSUNDEEP
pythoncnumpy
neekogi
neekogi

Reputation: 27

Numpy complicated data structure

I'm quite new to python. What I would like to do is reading some c-like structs from a binary file. The structs, in the c program that creates them, are defined as:

struct B{
    uint16 y;
    uint8 c[SIZE2];
}

struct A{
    uint32 x;
    struct B b[SIZE1];
}

I would like to be able to read all the A-structs using the NumPy package function fromFile, but I have no idea how to call a proper dtype method, like:

record = numpy.dtype([
    ("field1", numpy.uint8),
    ("field2", numpy.uint16),
    ("field3", numpy.uint32)
])

with a such complicated data structure.

Could you please help me writing it? Thank you in advance!

Upvotes: 2

Views: 103

Answers (1)

hpaulj
hpaulj

Reputation: 231605

This is a bit of guess, since I haven't worked with C structs much.

In [125]: SIZE1, SIZE2 = 3,4

In [127]: dtB=np.dtype([('y',np.uint16),('c',np.uint8,(SIZE2,))])
In [128]: np.ones((2,), dtype=dtB)
Out[128]: 
array([(1, [1, 1, 1, 1]), (1, [1, 1, 1, 1])], 
      dtype=[('y', '<u2'), ('c', 'u1', (4,))])
In [129]: _.itemsize
Out[129]: 6

In this definition, each record of this array consists of 6 bytes, 2 for y field, 4 for the c field.

Then nest that in a A definition

In [130]: dtA=np.dtype([('x',np.uint32),('b',dtB,(SIZE1,))])
In [131]: np.ones((2,), dtype=dtA)
Out[131]: 
array([(1, [(1, [1, 1, 1, 1]), (1, [1, 1, 1, 1]), (1, [1, 1, 1, 1])]),
       (1, [(1, [1, 1, 1, 1]), (1, [1, 1, 1, 1]), (1, [1, 1, 1, 1])])], 
      dtype=[('x', '<u4'), ('b', [('y', '<u2'), ('c', 'u1', (4,))], (3,))])
In [132]: _.itemsize
Out[132]: 22

Each record has 4 bytes for the x field, 3*6 for the 3 b elements.

In [133]: __.tobytes()
Out[133]: b'\x01\x00\x00\x00\x01\x00\x01\x01\x01\x01\x01\x00\x01\x01\x01\x01\x01\x00\x01\x01\x01\x01\x01\x00\x00\x00\x01\x00\x01\x01\x01\x01\x01\x00\x01\x01\x01\x01\x01\x00\x01\x01\x01\x01'

And trying to make the array more interesting:

In [136]: A['x']=[1,2]
In [139]: A['b']['y'] *= 3
In [141]: A['b']['c'][0]=2
In [142]: A['b']['c'][1]=3
In [143]: A
Out[143]: 
array([(1, [(3, [2, 2, 2, 2]), (3, [2, 2, 2, 2]), (3, [2, 2, 2, 2])]),
       (2, [(3, [3, 3, 3, 3]), (3, [3, 3, 3, 3]), (3, [3, 3, 3, 3])])], 
      dtype=[('x', '<u4'), ('b', [('y', '<u2'), ('c', 'u1', (4,))], (3,))])
In [144]: A[0].tobytes()
Out[144]: b'\x01\x00\x00\x00\x03\x00\x02\x02\x02\x02\x03\x00\x02\x02\x02\x02\x03\x00\x02\x02\x02\x02'

Are those bytestrings consistent with your c struct?

Upvotes: 1

Related Questions