CODEWITHSUNDEEP
pythonarraysnumpyundefined-behavior
John Zwinck
John Zwinck

Reputation: 249153

NumPy: Erroneous result when modifying array using itself

import numpy as np
bc = np.arange(10) # [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

# regular way using NumPy function
np.diff(bc) # array([1, 1, 1, 1, 1, 1, 1, 1, 1])

# something similar with array subtraction:
bc[1:] - bc[:-1] # array([1, 1, 1, 1, 1, 1, 1, 1, 1])

# but this does the wrong thing:
bc[1:] -= bc[:-1] # array([0, 1, 1, 2, 2, 3, 3, 4, 4, 5])

As a C and C++ programmer I can understand why this happens (it's memcpy() instead of memmove() all over again), but it seems like end users of Python and NumPy might not expect this. And I failed to find any documentation saying this wouldn't work.

The question is: is there a bug in NumPy here (probably not), or is there some NumPy documentation explaining what the rules are in situations like this, or is that missing from the documentation?

Secondly, I would like to figure out a safe, working solution which is quasi-optimal in space and time. That is, it should not allocate memory beyond a constant amount, and it should not be a silly pure-Python "for" loop. :) Those are the goals one might hope to achieve with the in-place modification which clearly doesn't work.

I'm using NumPy 1.8.0.

Upvotes: 4

Views: 120

Answers (3)

Fourier
Fourier

Reputation: 323

Im posting here again to correct my answer. perhaps try:

bc-=np.roll( bc , 1 )

#array([-9,  1,  1,  1,  1,  1,  1,  1,  1,  1])
#this arrays length is 10

#and then for an array with length 9
bc=bc[ 1 : ]  

#array([1,  1,  1,  1,  1,  1,  1,  1,  1])

sorry miss-understood the question before,

the reason for the result:

'array([0, 1, 1, 2, 2, 3, 3, 4, 4, 5])'

is

that this action is actually syntaxed as the deduction of 

let (say...) a=array([0,1,2,3,4,5,6,7,8,9])
updated a <- will be formed as [0, 1-0=1,2-1=1,3-1=2,4-2=2,5-2=3,6-3=3,7-3=4,8-4=4,9-4=5]

I think what you really want is above

tnx EdChum

Upvotes: 1

John Zwinck
John Zwinck

Reputation: 249153

I did find some discussion after posting this question. The important term to search for is "slice". It's here: http://numpy-discussion.10968.n7.nabble.com/Strange-behavior-in-setting-masked-array-values-in-Numpy-1-1-0-td11999.html

Partway down the page there is discussion about trying to detect and warn about this sort of thing, but it sounds like a lost cause. So I set about finding another in-place method to do what I wanted. Here it is!

bc[-1:0:-1] -= bc[-2::-1]

Credit to @fredtantini for explicitly writing out the pure-Python "for" loop which NumPy effectively does. That got me thinking about how to fix it in pure Python (iterate backward!), which led to the above solution.

Upvotes: 0

fredtantini
fredtantini

Reputation: 16556

I don't think that's a bug:
Doing bc[1:] -= bc[:-1] you are modifying the list while doing the operations.

The process is like

for i in range(1,len(bc)):
    bc[i] = bc[i] - bc[i-1]

So the next step bc[i-1] is modified:

"i"      1
bc[1:]:  1,2,3,4,5,…
bc[:-1]: 0,1,2,3,4,5,… 
bc[1]= 1 - 0 = 1
new bc: 0,1,2,3,4,5,…

"i"        2
bc[1:]:  1,2,3,4,5,…
bc[:-1]: 0,1,2,3,4,5,… 
bc[2]= 2 - 1 = 1
new bc: 0,1,1,3,4,5,…


"i"          3
bc[1:]:  1,1,3,4,5,…
bc[:-1]: 0,1,1,3,4,5,… 
bc[1]= 3 - 1 = 2
new bc: 0,1,1,2,4,5,…


"i"            4
bc[1:]:  1,1,2,4,5,…
bc[:-1]: 0,1,1,2,4,5,… 
bc[1]= 4 - 2 = 2
new bc: 0,1,1,2,2,5,…

And so on so forth.

For the rest of the question, I can't answer.

Upvotes: 0

Related Questions