CODEWITHSUNDEEP
pythondictionarymultiprocessing
Samos
Samos

Reputation: 111

Is it possible to parallelise creation of a dict in python?

I actually have this code to create a dict.

import multiprocessing
cpus = multiprocessing.cpu_count()
pool = multiprocessing.Pool(processes=cpus)
items = data['id'].unique()

def compute_weight(item):
    return sum(data[data['id'] == item])
weights = pool.map(compute_weight, items)

my_dict = {k, v for k, v in zip(items, weights)}

My question is: is there a way to directly create the dictionary with multiprocessing?

Upvotes: 0

Views: 41

Answers (1)

DeepSpace
DeepSpace

Reputation: 81614

No, but you don't need the loop.

Use

my_dict = dict(zip(items, weights))

which should be considerably faster because it pushes the loop down to the C level (in case of CPython)

Upvotes: 1

Related Questions