CODEWITHSUNDEEP
pythonmultithreadingpython-2.7processmultiprocessing
Priyanka Khairnar
Priyanka Khairnar

Reputation: 366

Python Multiprocessing on List of dictionaries

I have a list of dictionaries. list_of_dict = [{'name' : 'bob', 'weight': 50}, {'name' : 'ramesh', 'weight': 60}]

I want to process both the dictionaries at the same time

What should I use for this multiprocessing Pool or Process?

Upvotes: 5

Views: 1716

Answers (2)

Tanner Clark
Tanner Clark

Reputation: 691

I use the following because it is simple and easy to understand: 

from multiprocessing.dummy import Pool as ThreadPool

def threadfunction(dict):
    weight = dict.get('weight')
    return weight

pool = ThreadPool(5)
threadresults = pool.map(threadfunction,list_of_dict)
pool.close()
pool.join()

It will handle the mapping of the the list: list_of_dict. So you just need to pass in the number of threads to the ThreadPool() function and pass the function and list to the pool.map function. Simple!

Upvotes: 0

Priyanka Khairnar
Priyanka Khairnar

Reputation: 366

I have tried with Multiprocessing Pool

from multiprocessing.pool import ThreadPool as Pool

pool_size = 5 

def worker(item1, itme2):
    try:
        print(item1.get('weight'))
        print(itme2)
    except:
        print('error with item')

pool = Pool(pool_size)
items = [{'name' : 'bob', 'weight': 50}, {'name' : 'ramesh','weight': 60}]
for item in items:
    pool.apply_async(worker, (item, 'item2'))

pool.close()
pool.join()

Upvotes: 4

Related Questions