CODEWITHSUNDEEP
pythonarraysnumpydictionaryfor-loop
FaCoffee
FaCoffee

Reputation: 7909

Python: properly iterating through a dictionary of numpy arrays

Given the following numpy arrays:

import numpy
a=numpy.array([[1,1,1],[1,1,1],[1,1,1]])
b=numpy.array([[2,2,2],[2,2,2],[2,2,2]])
c=numpy.array([[3,3,3],[3,3,3],[3,3,3]])

and this dictionary containing them all:

mydict={0:a,1:b,2:c}

What is the most efficient way of iterating through mydict so to compute the average numpy array that has (1+2+3)/3=2 as values?

My attempt fails as I am giving it too many values to unpack. It is also extremely inefficient as it has an O(n^3) time complexity:

aver=numpy.empty([a.shape[0],a.shape[1]])

for c,v in mydict.values():
    for i in range(0,a.shape[0]):
        for j in range(0,a.shape[1]):
            aver[i][j]=mydict[c][i][j] #<-too many values to unpack

The final result should be:

In[17]: aver
Out[17]: 
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])

EDIT

I am not looking for an average value for each numpy array. I am looking for an average value for each element of my colleciton of numpy arrays. This is a minimal example, but the real thing I am working on has over 120,000 elements per array, and for the same position the values change from array to array.

Upvotes: 0

Views: 2088

Answers (2)

juanpa.arrivillaga
juanpa.arrivillaga

Reputation: 95957

You really shouldn't be using a dict of numpy.arrays. Just use a multi-dimensional array:

>>> bigarray  = numpy.array([arr.tolist() for arr in mydict.values()])
>>> bigarray
array([[[1, 1, 1],
        [1, 1, 1],
        [1, 1, 1]],

       [[2, 2, 2],
        [2, 2, 2],
        [2, 2, 2]],

       [[3, 3, 3],
        [3, 3, 3],
        [3, 3, 3]]])
>>> bigarray.mean(axis=0)
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])
>>>

You should modify your code to not even work with a dict. Especially not a dict with integer keys...

Upvotes: 1

DSM
DSM

Reputation: 353179

I think you're making this harder than it needs to be. Either sum them and divide by the number of terms:

In [42]: v = mydict.values()

In [43]: sum(v) / len(v)
Out[43]: 
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])

Or stack them into one big array -- which it sounds like is the format they probably should have been in to start with -- and take the mean over the stacked axis:

In [44]: np.array(list(v)).mean(axis=0)
Out[44]: 
array([[ 2.,  2.,  2.],
       [ 2.,  2.,  2.],
       [ 2.,  2.,  2.]])

Upvotes: 1

Related Questions