CODEWITHSUNDEEP
pythonpython-3.5memoryview
Jason
Jason

Reputation: 163

slice assignment slower using memoryview (Python 3.5.0)

I have a large bytearray x and want to assign a slice of it to a slice of another bytearray y

x = bytearray(10**7) #something else in practice
y = bytearray(6*10**6)
y[::6] = x[:2*10**6:2]

I figured using memoryview would be faster, and indeed

memoryview(x)[:2*10**6:2]

is very fast. However, 

y[::6] = memoryview(x)[:2*10**6:2]

takes 5 times as long as y[::6] = x[:2*10**6:2]

  1. Am I missing something, or is this slowdown a bug in Python?
  2. What is the fastest way to do this in Python (a) if I want to repeatedly assign a known number of 0's, and (b) in general?

Upvotes: 2

Views: 1019

Answers (1)

Dunes
Dunes

Reputation: 40713

The slowdown is not so much a bug, but that memoryview and the buffer protocol are still relatively new to and are poorly optimised. The underlying code to y[::6] = memoryview(x)[:2*10**6:2] creates a contiguous copy of the bytearray before copying it over. Meaning it will be slower than directly creating and assigning a normal slice of the bytearray. Indeed, in this particular instance (on my machine), using a memoryview is closer in speed to using y[::6] = islice(x, None, 2*10**6, 2) than direct assignment.

numpy has existed for much longer and is much better optimised for the types of operations you are interested in doing.

Using ipython:

In [1]: import numpy as np; from itertools import islice

In [2]: x = bytearray(10**7)

In [3]: y = bytearray(6*10**6)

In [4]: x_np = np.array(x)

In [5]: y_np = np.array(y)

In [6]: %timeit y[::6] = memoryview(x)[:2*10**6:2]
100 loops, best of 3: 10.9 ms per loop

In [7]: %timeit y[::6] = x[:2*10**6:2]
1000 loops, best of 3: 1.65 ms per loop

In [8]: %timeit y[::6] = islice(x, None, 2*10**6, 2)
10 loops, best of 3: 22.9 ms per loop

In [9]: %timeit y_np[::6] = x_np[:2*10**6:2]
1000 loops, best of 3: 911 µs per loop

The last two have the added benefit of having very little memory overhead.

Upvotes: 1

Related Questions