CODEWITHSUNDEEP
pythonparallel-processingmpi4py
Sam He
Sam He

Reputation: 79

mpi4py passing dict object

#mpiexec -n 3 python pass_dict.py
from mpi4py import MPI
import psycopg2
comm = MPI.COMM_WORLD
rank = comm.Get_rank()
size = comm.Get_size()

tax_dict={}
if rank == 0:
    tax_files=['2008','2009','2011','2012','2013','2014','2015']
    file_dir='/zhome/nah316/taxonomy/'
    for tax_file in tax_files:
        filename=file_dir + tax_file+'.csv'
        with open(filename,'r') as f:
            temp={}
            for line in f:
                temp_list=[]
                splitted_line = line.split()
                tag=splitted_line[1]
                temp_list.append(tag)
                temp[splitted_line[1]] = temp_list
            tax_dict[tax_file]=temp
else:
    tax_dict=None

comm.bcast(tax_dict, root = 0)

print  '-' * 20, rank , '-'* 30
print tax_dict['2015']['InvestmentSoldNotYetPurchasedRestrictedCost']

Here's the code that I tried to construct a dictionary of dictionary, and bcast it on the communicator to 2 other core. when I ran it I get the error:

-------------------- 2 ------------------------------
Traceback (most recent call last):
  File "pass_dict.py", line 33, in <module>
    print tax_dict['2015']['InvestmentSoldNotYetPurchasedRestrictedCost']
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- 1 ------------------------------
Traceback (most recent call last):
  File "pass_dict.py", line 33, in <module>
    print tax_dict['2015']['InvestmentSoldNotYetPurchasedRestrictedCost']
TypeError: 'NoneType' object has no attribute '__getitem__'
-------------------- 0 ------------------------------
['InvestmentSoldNotYetPurchasedRestrictedCost']

it seems to me that the dictionary that got passed to core other than the root has lost some functionality as a dictionary. why is this? How should I go around this and make the pass as it is on the root node?

Thanks in advance!

Upvotes: 1

Views: 1364

Answers (1)

David Henty
David Henty

Reputation: 1764

I don't know much about python or mpi4py in detail, but I found some code at https://mpi4py.scipy.org/docs/usrman/tutorial.html that would imply that you need to assign the result of comm.bcast to the dictionaries on the other ranks.

The code should be

tax_dict = comm.bcast(tax_dict, root = 0)

Maybe this solves the problem?

Upvotes: 2

Related Questions