CODEWITHSUNDEEP
python
Mayukh Sarkar
Mayukh Sarkar

Reputation: 2625

Why remove is not working on list of dicts?

I have the following code. 

import os

products =  [

        {"Product": "S65-85OS04_M2M_GP211_JC222_R6",
         "PlatformName": "winPc",
         "PlatformVariant": "eeprom",
         "DocGeneration": False,
         "BuildStatus": "Pass",
        },

        {"Product": "SC5-01OS19_GP221_JC302_LTE_MTN",
         "PlatformName": "winPc",
         "PlatformVariant": "flash",
         "DocGeneration": False,
         "BuildStatus": "Fail",
         },

        {"Product": "SC5-01OS01_GP211_JC302_LTE_TMO",
         "PlatformName": "winPc",
         "PlatformVariant": "flash",
         "DocGeneration": False,
         "BuildStatus": "Pass",
         }
    ]

class UTE(object):

    def __init__(self, workspace, products, blackList=None):
        for each in products:
            # print each
            if each['Product'] in blackList:
                products.remove(each)
        for each in products:
            print each["Product"]

if __name__ == '__main__':
    ins = UTE('ws', products, ["SC5-01OS01_GP211_JC302_LTE_TMO", "SC5-01OS19_GP221_JC302_LTE_MTN"])

Now whenever I run this, it removes only one entry in the dict. For example, in this case it is removing the 2nd entry and that is SC5-01OS19_GP221_JC302_LTE_MTN. I believe this is something related to shallow copy..Am I right ?? If not then how solve this issue?

Upvotes: 0

Views: 60

Answers (3)

AKS
AKS

Reputation: 19861

You are removing the entry from the list while iterating over the same list. You could simply use list comprehension to keep the elements which are required:

products = [product for product in products 
            if product.get('Product', None) not in blacklist]

Or using filter:

products = filter(lambda product: product.get('Product', None) not in blacklist, products)

Update regarding remove from a = [1, 2, 3, 4, 5, 6, 7, 8]:

You say following code works:

a = [1, 2, 3, 4, 5, 6, 7, 8]
for e in a:
    if e % 2 = 0:
        a.remove(e)

print(a) # [1, 3, 5, 7]

Well, it doesn't work but it is just an illusion, let's add some print statements to understand:

for e in a:
    print 'Next Item: ', e
    if e % 2 = 0:
        a.remove(e)

    print 'Current List: ', a

And, here is the output from the print statements:

Next Item:  1
Current List:  [1, 2, 3, 4, 5, 6, 7, 8]
Next Item:  2
Current List:  [1, 3, 4, 5, 6, 7, 8]
Next Item:  4
Current List:  [1, 3, 5, 6, 7, 8]
Next Item:  6
Current List:  [1, 3, 5, 7, 8]
Next Item:  8
Current List:  [1, 3, 5, 7]

As, you can see, just before the 3rd iteration the a is updated and 3 is shifted to the 2nd position and since 2nd position is already iterated 3 never comes in the loop. And, similarly for others so the loop doesn't run 8 times but only 5 times.

If you change the order of values in a you won't get the same behavior:

a = [1, 4, 2, 3, 6, 8, 7, 5]

Now, run the above `for loop again:

Next Item:  1
Current List:  [1, 4, 2, 3, 6, 8, 7, 5]
Next Item:  4
Current List:  [1, 2, 3, 6, 8, 7, 5]
Next Item:  3
Current List:  [1, 2, 3, 6, 8, 7, 5]
Next Item:  6
Current List:  [1, 2, 3, 8, 7, 5]
Next Item:  7
Current List:  [1, 2, 3, 8, 7, 5]
Next Item:  5
Current List:  [1, 2, 3, 8, 7, 5]

So here at the end of the list the value of a is [1, 2, 3, 8, 7, 5].

So like I said earlier: It doesn't work but it is just an illusion that it does

Upvotes: 1

theorifice
theorifice

Reputation: 690

As explained by AKS, you can use list comprehensions to filter out list elements. In addition, you can use the built in filter function:

some_integers = range(15)

# via filter
odd_integers = filter(lambda i: i%2 != 0, some_integers)

# via list comprehension
odd_integers = [num for num in some_integers if num %2 != 0]

Ultimately, the issue you're running into is that you are modifying your list as you iterate through it. This has been discussed many times, for example: Modifying list while iterating

Upvotes: 0

Yu Hao
Yu Hao

Reputation: 122493

for each in products:
    # print each
    if each['Product'] in blackList:
        products.remove(each)

Here you are modifying the list while iterating over it. That would give you unexpected result. A quick fix is: iterate over a copy of the list:

for each in products[:]:

Upvotes: 1

Related Questions