CODEWITHSUNDEEP
pythonmultiprocessing
Nunes
Nunes

Reputation: 61

imap unordered values access

I have the following situation which illustrates a more complex case.

Dependencies:

#formula
def cube_rand(x):

  c = x**3
  time.sleep(random.random())

  return (x,c)

#dataframe to be used
dict_input = {
              'col1': ['a','b','c'],
              'col2': [1,2,3]
             }

df_test = pd.DataFrame(dict_input)

imap instance:

from multiprocessing import Pool

start = time.time()

p=Pool(8)

results = p.imap_unordered(cube_rand, (index for index,row in df_test.iterrows()))

print(results)

done = time.time()

print("processing time: ", done-start)

I would like to print the pair ("index","index^3"), unordered, for each of the rows in the defined dataframe. However, I receive the following output:

<multiprocessing.pool.IMapUnorderedIterator object at 0x7fb5af0cbd50>
processing time:  0.15177679061889648

Any guesses on how to work around this problem?

Thanks

Upvotes: 0

Views: 217

Answers (1)

Abstract
Abstract

Reputation: 995

You'll want to iterate over the results of the imap_unordered call:

for x in p.imap_unordered(cube_rand, (index for index,row in df_test.iterrows())):
    print(x)

Output:

(0, 0)
(2, 8)
(1, 1)
processing time:  1.1946678161621094

Upvotes: 1

Related Questions