CODEWITHSUNDEEP
pythonnumpy
pythonic
pythonic

Reputation: 109

Appending to a specific row in a numpy array

I am trying to append a number to a particular row of a 2D numpy array. I did the following but it doesn't work right:

onspiketimes=np.array([[] for i in range(9)])
neurs = [3, 4, 8, 2]
onspiketimes[neurs]=2
>>> onspiketimes
array([], shape=(9, 0), dtype=float64)

As you can see that nothing got appended,onspiketimes still remains empty. However I could do this with list of lists as below:

>>> onspiketimes= [[] for i in range(9)]
>>> for items in neurs:
        onspiketimes[items].append(2)
>>> for items in neurs:
        onspiketimes[items].append(3)
>>> onspiketimes
[[], [], [2, 3], [2, 3], [2, 3], [], [], [], [2, 3]]

This is a contrived example, I can't really compare numpy and lists here but in my real applications for loops will really effect the speedup. I was hoping to solve this using array slicing in numpy.

Upvotes: 1

Views: 1437

Answers (2)

Daniel Ong
Daniel Ong

Reputation: 283

NumPy arrays are stored in a contiguous block of memory so appending new elements into an array the way you've described would require allocating a new, bigger block of memory, copying over the existing elements in the array, and constructing the new, modified array. This is inefficient and undesirable. For what you're trying to do in your example, a list of lists is much better than a NumPy array.

Upvotes: 1

hpaulj
hpaulj

Reputation: 231325

To do the same thing with an array, you have to make an object dtype array, and initialize it with [].

In [580]: alist = [[] for _ in range(10)]
In [581]: arr = np.empty(10, object)
In [582]: arr[:]= alist
In [583]: arr
Out[583]: 
array([list([]), list([]), list([]), list([]), list([]), list([]),
       list([]), list([]), list([]), list([])], dtype=object)
In [584]: neurs = [3, 4, 8, 2]
In [585]: for item in neurs:
     ...:     arr[item].append(2)
     ...:     
In [586]: for item in neurs:
     ...:     arr[item].append(3)
     ...:     
In [587]: 
In [587]: arr
Out[587]: 
array([list([]), list([]), list([2, 3]), list([2, 3]), list([2, 3]),
       list([]), list([]), list([]), list([2, 3]), list([])], dtype=object)

In [588]: arr.tolist()
Out[588]: [[], [], [2, 3], [2, 3], [2, 3], [], [], [], [2, 3], []]

Iterating on an object array is a little slower than iterating on a list (but faster than iterating on the rows of a 2d array.

Doing something similar with a 2d numeric array:

In [589]: arr2 = np.zeros((10,5),int)
In [590]: arr2[neurs, 0] = 2
In [591]: arr2[neurs, 1] = 3
In [592]: arr2
Out[592]: 
array([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0],
       [2, 3, 0, 0, 0],
       [0, 0, 0, 0, 0]])

This will be faster, but the result isn't a ragged array.

What you are doing with lists can't be done (exactly) faster with arrays. It doesn't fit the multidimensional array model that has many fast compiled building blocks.

You ask about map. That is similar to a list comprehension, in function and speed. It produces a new list. It shouldn't be used for side-effects, like appending to values to an existing list.

Upvotes: 1

Related Questions