CODEWITHSUNDEEP
pythonparallel-processingmpipython-3.4mpi4py
abinitio
abinitio

Reputation: 805

Nesting mpi calls with mpi4py

I am trying to use mpi4py to call a second instance of an mpi executable.

I am getting the error:

Open MPI does not support recursive calls of mpirun

But I was under the impression that is exactly what Spawn is supposed to be able to handle - i.e. setting up a new communicator within which another mpi command could be launched.

The test code:

parent.py:

#!/usr/bin/env python
from mpi4py import MPI
import numpy
import sys


rank = MPI.COMM_WORLD.Get_rank()
new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
print(new_comm.Get_rank())
new_comm.Spawn(sys.executable,
                           args=['test.py'],
                           maxprocs=4)

which calls test.py:

#!/usr/bin/env python
from mpi4py import MPI
import numpy
import os
import sys

comm = MPI.Comm.Get_parent()
rank = comm.Get_rank()

cwd=os.getcwd()
directory=os.path.join(cwd,str(rank))
os.chdir(directory)


os.system('{}'.format('mpirun -np 4 SOME_MPI_EXECUTABLE_HERE'))



print("Finished in "+directory)
os.chdir(cwd)

comm.Disconnect()

I'm running with:

mpirun --oversubscribe -np 1 parent.py

Using openmpi 2.0.0 with gcc, and python/3.4.2

Anyone have any bright ideas as to why this is happening.....

Thanks!

Upvotes: 2

Views: 2190

Answers (1)

abinitio
abinitio

Reputation: 805

The following code seems to perform the way I wanted.

#!/usr/bin/env python
from mpi4py import MPI
import numpy
import sys
import os

rank = MPI.COMM_WORLD.Get_rank()
new_comm = MPI.COMM_WORLD.Split(color=rank, key=rank)
print(new_comm.Get_rank())

cwd=os.getcwd()
os.mkdir(str(rank))
directory=os.path.join(cwd,str(rank))
print(rank,directory)
os.chdir(directory)


new_comm.Spawn("SOME_MPI_EXECUTABLE_HERE",
                  args=[""],
                           maxprocs=4)

run with:

mpirun --oversubscribe -np 4 parent.py

Seems to start 4 instances of SOME_MPI_EXECUTABLE each running on 4 cores.

(Thanks to Zulan)

Upvotes: 3

Related Questions