Interpreting a bytearray as array of longs in Python 2.7

Question

In Python 3 it is possible to interpret the underlying memory as array of bytes or ints or longs via memoryview.cast():

[] b=bytearray(2*8)
[] b
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00')
[] m=memoryview(b).cast('L') #reinterpret as an array of unsigned longs
[] m[1]=2**64-1
[] b
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff')

As it can be seen, we can access the bytearray b as if it where an array of unsigned longs (which are 8 byte long on my machine) with help of the memoryview m.

However, in Python 2.7 the memoryview lacks the method cast.

Thus my question: Is there a possibility to reinterpret a bytearray as an array of longs in Python 2.7?

It is important to do it without copying/allocating more memory.

Time needed for reading a long value via T[i] on my machine (Python 3.4):

python list:                40ns  (fastest but needs 24 bytes per element)
python array:              120ns  (has to create python int-object)
memoryview of bytearray    120ns  (the same as array.array)
Jean-François's solution: 6630ns  (ca. 50 times slower)
Ross's solution:           120ns

Answer 1

You can use ctypes and its from_buffer method to create a ctypes array of unsigned longs that shares its memory with the bytearray object.

For example:

import ctypes

ba = bytearray(b'\x00' * 16)
a = (ctypes.c_ulong * (len(ba) / 8)).from_buffer(ba)
a[1] = -1
print repr(ba)

Answer 2

Not really a reinterpretation since it creates a copy. In that case, you would have to

• convert bytearray to list of longs
• work with the list of longs
• convert back to bytearray when you're done

(a bit like working with str characters when converting them to list of characters, modifying them, then joining them back to a string)

import struct

b=bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff')

larray=[]

for i in range(0,len(b),8):
    larray.append(struct.unpack('@q',b[i:i+8])[0])

print(larray)

larray[1]=1000

b = bytearray()

for l in larray:
    b += struct.pack('@q',l)

print(b)

Method which does not involve copy (only works for long ints):

def set_long(array,index,value):
    index *= 8

    if sys.byteorder=="little":
        shift=0
        for i in range(index,index+8):
            array[i] = (value>>shift) & 0xFF
            shift += 8
    else: # sys.byteorder=="big"
        shift = 56
        for i in range(index+8,index,-1):
            array[i] = (value<<shift) & 0xFF
            shift -= 8

def get_long(array,index):
    index *= 8
    value = 0

    if sys.byteorder=="little":
        shift=0
        for i in range(index,index+8):
            c = (array[i]<<shift)
            value += c
            shift += 8
    else: # sys.byteorder=="big"
        shift = 56
        for i in range(index+8,index,-1):
            value += (array[i]>>shift)
            shift -= 8

    return value

b=bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\xff\xff\xff\xff')    
print(get_long(b,1)==2**64-1)
set_long(b,1,2001)
print(b)

print(get_long(b,1))

output:

bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\xe8\x03\x00\x00\x00\x00\x00\x00')
True
bytearray(b'\x00\x00\x00\x00\x00\x00\x00\x00\xd1\x07\x00\x00\x00\x00\x00\x00')
2001