CODEWITHSUNDEEP
pythonarraysnumpy
Timmmm
Timmmm

Reputation: 97078

Weird behaviour with numpy array operations

Explain this:

>>> a = np.arange(10)
>>> a[2:]
array([2, 3, 4, 5, 6, 7, 8, 9])
>>> a[:-2]
array([0, 1, 2, 3, 4, 5, 6, 7])
>>> a[2:] - a[:-2]
array([2, 2, 2, 2, 2, 2, 2, 2])
>>> a[2:] -= a[:-2]
>>> a
array([0, 1, 2, 2, 2, 3, 4, 4, 4, 5])

The expected result is of course array([0, 1, 2, 2, 2, 2, 2, 2, 2, 2]).

I'm going to guess this is something to do with numpy parallelising things and not being smart enough to work out that it needs to make a temporary copy of the data first (or do the operation in the correct order).

In other words I suspect it is doing something naive like this:

for i in range(2, len-2):
    a[i] -= a[i-2]

For reference it works in Matlab and Octave:

a = 0:9
a(3:end) = a(3:end) - a(1:end-2)

a =

  0  1  2  3  4  5  6  7  8  9

a =

  0  1  2  2  2  2  2  2  2  2

And actually it works fine if you do:

a[2:] = a[2:] - a[:-2]

So presumably this means that a -= b is not the same as a = a - b for numpy!

Actually now that I come to think of it, I think Mathworks gave this as one of the reasons for not implementing the +=, -=, /= and *= operators!

Upvotes: 5

Views: 238

Answers (2)

bogatron
bogatron

Reputation: 19159

The unexpected behavior is due to array aliasing because (as @JoshAdel stated in his answer), slicing returns a view, rather than a copy of the array. Your example of the "naive" loop already explains how the result is computed. But I'll add two points to your explanation:

First, the unexpected behavior is not due to numpy parallelizing operations. If the operation were parallelized, then you shouldn't expect to [consistently] see the result of the naive loop (since that result depends on ordered execution of the loop). If you repeat your experiment several times - even for large arrays - you should see the same result.

Second, while your presumption is true in general, I would state it this way:

a -= b is the same as a = a - b for two numpy arrays when a and b are not aliased.

Upvotes: 1

JoshAdel
JoshAdel

Reputation: 68742

When you slice a numpy array as you are doing in the example, you get a view of the data rather than a copy.

See:

http://scipy-lectures.github.io/advanced/advanced_numpy/#example-inplace-operations-caveat-emptor

Upvotes: 4

Related Questions