CODEWITHSUNDEEP
pythonarraysnumpyindexing
Will Martin
Will Martin

Reputation: 1023

Numpy sub-array assignment with advanced, mixed indexing

Original question

I am getting a very odd error message when I try to assign some of the elements of an array. I am using a combination of a slice and a set of indices. See the following simple example.

 import scipy as sp

 a = sp.zeros((3, 4, 5))
 b = sp.ones((4, 5))

 I = sp.array([0, 1, 3])

 b[:, I] = a[0, :, I]

This code raises the following ValueError:

ValueError: shape mismatch: value array of shape (3,4) could not be broadcast to indexing result of shape (3,4)

--

Follow up

Be careful when using a combination of a slice and seq. of integers. As pointed out on github:

x = rand(3, 5, 7)

print(x[0, :, [0,1]].shape)
# (2, 5)

print(x[0][:, [0,1]].shape)
# (5, 2)

This is how numpy is designed to work, but it is nevertheless a bit confusing that x[0][:, I] is not the same as x[0, :, I]. Since this is the behavior I want I choose to use x[0][:, I] in my code.

Upvotes: 5

Views: 9177

Answers (3)

hpaulj
hpaulj

Reputation: 231385

Looks like there are some errors in copying your code to question.

But I suspect there's a known problem with indexing:

In [73]: a=np.zeros((2,3,4)); b=np.ones((3,4)); I=np.array([0,1])

Make I 2 elements. Indexing b gives the expected (3,2) shape. 3 rows from the slice, 2 columns from I indexing

In [74]: b[:,I].shape
Out[74]: (3, 2)

But with 3d a we get the transpose.

In [75]: a[0,:,I].shape
Out[75]: (2, 3)

and assignment would produce an error

In [76]: b[:,I]=a[0,:,I]
...
ValueError: array is not broadcastable to correct shape

It's putting the 2 element dimension defined by I first, and the 3 element from : second. It's a case of mixed advanced indexing that has been discussed earlier - and there's a bug issue as well. (I'll have to look those up).

You are probably using a newer numpy (or scipy) and getting a different error message.

It's documented that indexing with two arrays or lists, and slice in the middle, puts the slice at the end, e.g.

In [86]: a[[[0],[0],[1],[1]],:,[0,1]].shape
Out[86]: (4, 2, 3)

The same thing is happening with a[0,:,[0,1]]. But there's a good argument that it shouldn't be this way.

As to a fix, you could transpose a value, or change the indexing

In [88]: b[:,I]=a[0:1,:,I]

In [90]: b[:,I]=a[0,:,I].T

In [91]: b
Out[91]: 
array([[ 0.,  0.,  1.,  1.],
       [ 0.,  0.,  1.,  1.],
       [ 0.,  0.,  1.,  1.]])

In [92]: b[:,I]=a[0][:,I]

https://github.com/numpy/numpy/issues/7030

https://github.com/numpy/numpy/pull/6256

Upvotes: 3

heltonbiker
heltonbiker

Reputation: 27575

Here I get this error with indices [0,1,4]:

IndexError: index 4 is out of bounds for axis 2 with size 4

Its suggesting the value 4 is being used as an index, while the SIZE 4 implies the max index would be 3.

EDIT: now that you changed it to [0, 1, 3], it's running fine here.

EDIT: with your current code, I get the same error, but when I print the arrays themselves, they have a transverse shape:

print b[:, I]
print a[0, :, I]

[[ 1.  1.  1.]
 [ 1.  1.  1.]
 [ 1.  1.  1.]
 [ 1.  1.  1.]]

[[ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]
 [ 0.  0.  0.  0.]]

Upvotes: 1

thecardkid
thecardkid

Reputation: 999

First of all it looks like you're missing a comma on the line 6:

I = sp.array([0,1,4])

Secondly, I would expect the value 4 in the array I to raise an IndexError, since both a and b have a max dimension of 4. I suspect you might want:

I = sp.array([0,1,3])

Making these changes run the program for me, and I got b as:

[[ 0.  0.  1.  0.]
 [ 0.  0.  1.  0.]
 [ 0.  0.  1.  0.]]

Which I suspect is what you want.

Upvotes: 1

Related Questions