CODEWITHSUNDEEP
pythonpython-2.7
twfx
twfx

Reputation: 1694

improve iteritems performance or a better way to loop through python dictionary

It takes approx. 190s to complete the loop below on a laptop with Intel i5, 8GB, Windows 7.

count = 0
for l in xrange(NUM_ROOM): 
    for k in xrange(NUM_SLOT):
        for m in xrange(NUM_ACTIVITY):
            count = count+1

            for dk, [om, ol, ok] in xDict.iteritems(): 
                if dk == (m,l,k):
                    # do something

I did a trace and found out that the total loop is 13960 x 19040 ~ 2bils.

len(self.xDict): 13960
count: 19040

xDict looks something like this (an example):

xDict = {(19, 4, 118): [19, 4, 4], (4, 12, 25): [4, 12, 7], (15, 19, 121): [15, 19, 21], (23, 5, 219): [23, 5, 9], (13, 5, 19): [13, 5, 1]}

I am trying to get the value of a key in the dictionary. Is there anyway to improve the performance?

Sorry, I think I know what happen. 

count = 0
for l in xrange(NUM_ROOM): 
    for k in xrange(NUM_SLOT):
        for m in xrange(NUM_ACTIVITY):
            count = count+1

            val = xDict.get((m,l,k))
            if val != None:  
                [om, ol, ok] = val  
                # do something

Now it takes 1s.

Upvotes: 0

Views: 315

Answers (1)

Martijn Pieters
Martijn Pieters

Reputation: 1121992

Don't loop over all items just to find one key. Use a membership test instead:

if (m, l, k) in xDict:
    om, ol, ok = xDict[m, l, k]

This removes the need to loop over 13960 items for each of your 19040 iterations.

You haven't told us much about what you are trying to achieve here, if you are using m, l and k for something else too, but consider just looping over the keys instead; that way you'd reduce this to a loop over 13960 items instead of 19040 iterations.

Upvotes: 1

Related Questions