CODEWITHSUNDEEP
pythonpython-3.xcython
L3viathan
L3viathan

Reputation: 27283

Assign bytes to Cython memory view

I have a memory view of a 2-dimensional Cython array defined as

cdef unsigned char[:,:] store = view.array(
    shape=(n, 141),
    itemsize=sizeof(unsigned char),
    format="B",
)

, where n is the number of "rows" I have. For the sake of the example, lets assume n=2. Each row is a Python bytes object of size 141, which I have in a list of bytes:

rows = [
    b"4612edc38e7acc81520860b210c4ae7a14ace72bbb710514bd5ef71bb96e7af55b307580b015efac3cb84339fa9ff99401ed6fe211bb75b937fa575fc3928500c511aee084e5c",
    b"beb71c68b420fcb093d7c5894d726239bb0620229e46f91c11d3c2cd688e37c0c279ec844152cf87ea6c3f6915ac90aeb83ade44dfe22b1ae9c6fdd87e813eed98e5faca4d250",
]

I now want to move the data inside these rows into the memory view. At the moment, I'm doing this in a rather stupid way:

for i, row in enumerate(rows):
    for j, byte_ in enumerate(row):
        store[i, j] = byte_

While this works, I assume it is inefficient (unless cythonize does some magic to get rid of the loops), and it just looks messy and overly complicated. I want to be able to simply do something like:

store[:,:] = rows

or, at least:

for i, row in enumerate(rows):
    store[i, :] = row # fails with TypeError: an integer is required

Sadly, none of that works. I also tried the following:

# fails because the resulting number is a bigint (?)
store[i, :] = int.from_bytes(bytearray, sys.byteorder)
# fails with TypeError: an integer is required
store[i, :] = np.array(list(row))

There must be a better way. What am I missing?

Upvotes: 1

Views: 473

Answers (1)

DavidW
DavidW

Reputation: 30899

The following works:

cdef const unsigned char[::1] row

for i, row in enumerate(rows):
    store[i, :] = row

Essentially it can only do a sliced "element-by-element" copy if it knows that row is a memoryview. Otherwise it assumes that row is a single value that the slice should be filled with.

I've used const unsigned char[::1] since bytes objects are immutible and contiguous. Without the const it won't compile, while the ::1 just makes it a little more efficient.

Upvotes: 3

Related Questions