CODEWITHSUNDEEP
pythonlistdictionary
Alan
Alan

Reputation: 559

How to unpack keys with list of values to multiple dictionaries to a list without overwriting?

I have a list of dictionaries:

data = [
    {'name': 'foo', 'scores': [2]},
    {'name': 'bar', 'scores': [4, 9, 3]},
    {'name': 'baz', 'scores': [6, 1]}
]

I want to create a new list which has each individual score separated out like this:

list = [
    {'name': 'foo', 'scores': [2], 'score': 2},
    {'name': 'bar', 'scores': [4, 9, 3], 'score': 4},
    {'name': 'bar', 'scores': [4, 9, 3], 'score': 9},
    {'name': 'bar', 'scores': [4, 9, 3], 'score': 3},
    {'name': 'baz', 'scores': [6, 1], 'score': 6},
    {'name': 'baz', 'scores': [6, 1], 'score': 1}
]

I can then loop through each row, and each score, to create a new dictionary:

for row in data:
    scores = row['scores']  # list of values
    for score in scores:
        new_row = row
        new_row['score'] = score
        print(new_row)

Which gives me exactly what I want:

{'name': 'foo', 'scores': [2], 'score': 2}
{'name': 'bar', 'scores': [4, 9, 3], 'score': 4}
{'name': 'bar', 'scores': [4, 9, 3], 'score': 9}
{'name': 'bar', 'scores': [4, 9, 3], 'score': 3}
{'name': 'baz', 'scores': [6, 1], 'score': 6}
{'name': 'baz', 'scores': [6, 1], 'score': 1}

However, I'm having trouble adding these dictionaries to a list. When I use the append() function to add each dictionary to a new list:

list = []

for row in data:
    scores = row['scores']  # list of values
    for score in scores:
        new_row = row
        new_row['score'] = score
        list.append(new_row)

    print(list)

It seems to overwrite some of the previous items:

[
{'name': 'foo', 'scores': [2], 'score': 2},
{'name': 'bar', 'scores': [4, 9, 3], 'score': 3},
{'name': 'bar', 'scores': [4, 9, 3], 'score': 3},
{'name': 'bar', 'scores': [4, 9, 3], 'score': 3},
{'name': 'baz', 'scores': [6, 1], 'score': 1},
{'name': 'baz', 'scores': [6, 1], 'score': 1}
]

What's going on here? Why is it printing the rows correctly, but overwriting previous items when adding to a list? I thought append() simply adds new items to the end of a list without altering other items?

Upvotes: 3

Views: 1080

Answers (3)

kmario23
kmario23

Reputation: 61335

How about a simple list comprehension, to achieve all these in a single step:

In [269]: [{**d, **{'score': v}} for d in data for v in d['scores']]
Out[269]: 
[{'name': 'foo', 'score': 2, 'scores': [2]},
 {'name': 'bar', 'score': 4, 'scores': [4, 9, 3]},
 {'name': 'bar', 'score': 9, 'scores': [4, 9, 3]},
 {'name': 'bar', 'score': 3, 'scores': [4, 9, 3]},
 {'name': 'baz', 'score': 6, 'scores': [6, 1]},
 {'name': 'baz', 'score': 1, 'scores': [6, 1]}]

Explanation/Clarification:

This list comprehension does what OP finally needs. We start by iterating over each dictionary in our list of dictionaries data and for each value v in current dictionary's scores with this nested for loop,

for d in data for v in d['scores']  # order goes from left to right

we add a key score and a value v by unpacking and then we also unpack the current dictionary since OP needs that as well. At the end we concatenate both of these using {**d, **{'score': v}} and that's what we need to achieve.

The concatenation is done using { } or dict() because we unpack the keys and values from both d and {'score': v}; Thus, an alternative is:

In [3]: [dict(**d, **{'score': v}) for d in data for v in d['scores']]
Out[3]: 
[{'name': 'foo', 'score': 2, 'scores': [2]},
 {'name': 'bar', 'score': 4, 'scores': [4, 9, 3]},
 {'name': 'bar', 'score': 9, 'scores': [4, 9, 3]},
 {'name': 'bar', 'score': 3, 'scores': [4, 9, 3]},
 {'name': 'baz', 'score': 6, 'scores': [6, 1]},
 {'name': 'baz', 'score': 1, 'scores': [6, 1]}]

For more details on dictionary unpacking examples, please refer peps/pep-0448/

Upvotes: 4

Juliecodestack
Juliecodestack

Reputation: 9

The answers above are great. Thanks! Here I just explain the reason of the bug in a simple way. I added two print():

for score in scores:
        print(row)
        new_row = row
        new_row['score'] = score
        list.append(new_row)
        print(list)

part of the results:

......
{'name': 'bar', 'scores': [4, 9, 3]}
[{'name': 'foo', 'scores': [2], 'score': 2}, {'name': 'bar', 'scores': [4, 9, 3], 'score': 4}]
{'name': 'bar', 'scores': [4, 9, 3], 'score': 4}
[{'name': 'foo', 'scores': [2], 'score': 2}, {'name': 'bar', 'scores': [4, 9, 3], 'score': 9}, {'name': 'bar', 'scores': [4, 9, 3], 'score': 9}]
{'name': 'bar', 'scores': [4, 9, 3], 'score': 9}
[{'name': 'foo', 'scores': [2], 'score': 2}, {'name': 'bar', 'scores': [4, 9, 3], 'score': 3}, {'name': 'bar', 'scores': [4, 9, 3], 'score': 3}, {'name': 'bar', 'scores': [4, 9, 3], 'score': 3}]
......

So now we can see whennew_row = row, they refer to the same object. When new_row changes, row also changes. The list result is the result of the last loop for each scores.

Upvotes: 0

Miguel Garcia
Miguel Garcia

Reputation: 356

Here new_row always reference the current row object, that is the same for every score in this row object. You need to create a new object copying the current row. Use deepcopy from the copy package.

from copy import deepcopy
for row in data:
    scores = row['scores']  # list of values
    for score in scores:
        new_row = deepcopy(row)
        ...

Upvotes: 4

Related Questions