CODEWITHSUNDEEP
pythonpython-3.x
XO39
XO39

Reputation: 483

Add a list of values to an existing list of dictionaries (performance-critical)

How to add a list of value like this one:

l = [[1, 2], [2, 3], [3, 4]]

to an existing list of dictionaries like this:

d = [{'id': 1, 'type': 'normal'},
     {'id': 2, 'type': 'low'},
     {'id': 3, 'type': 'hight'}]

so that the end results will be like this:

d = [{'id': 1, 'type': 'normal', 'groups': [1, 2]},
     {'id': 2, 'type': 'low', 'groups': [2, 3]},
     {'id': 3, 'type': 'hight', 'groups': [3, 4]}]

Both list are of same size, and the index in both lists must match.

Performance is critical.

Thanks

Upvotes: 4

Views: 145

Answers (3)

Nathan Vērzemnieks
Nathan Vērzemnieks

Reputation: 5603

It appears to be quite a bit faster going with an approach that doesn't create any new lists or dictionaries or use indexing. This is the fastest I've found (and, in my opinion, the simplest):

for sub_d, groups in zip(d, l):
    sub_d['groups'] = groups

In my tests this is about three as fast as d = [{**s, 'groups': g} for s, g in zip(d, l)] - 0.085s for a list of 1000000 dictionaries vs 0.25s - and about twice as fast as the for i in range(len(d)) approach. It also has the advantage over the former - although in some cases it might be a disadvantage - that since it updates the dictionaries in place you can update a long list of them without creating a new list.

Two things I think are worth noting, though: one, the performance of different approaches in general often varies significantly depending on the exact structure of your data and the rest of your code; and two, the performance of any given bit of code is often much less critical than you might think. Is a difference of microseconds really critical in your code? Or are you dealing with millions of dictionaries and seconds matter? Personally, I think it's generally too soon to talk a lot about optimization before your code is more or less complete and you've determined it's too slow and you've profiled it in realistic circumstances.

Upvotes: 3

blhsing
blhsing

Reputation: 106881

You can zip the list of dicts and the list of lists in a list comprehension that outputs dicts with the sub-dicts unpacked and the sub-lists added to the groups key:

d = [{**s, 'groups': g} for s, g in zip(d, l)]

Upvotes: 1

R4444
R4444

Reputation: 2114

This works for me:

for i in range(len(d)):
  d[i]["groups"] = l[i]

Gives me:

[{'id': 1, 'type': 'normal', 'groups': [1, 2]}, {'id': 2, 'type': 'low', 'groups': [2, 3]}, {'id': 3, 'type': 'hight', 'groups': [3, 4]}]

In time:

1.9000000000005124e-05 seconds

Upvotes: 1

Related Questions