CODEWITHSUNDEEP
pythonmpijupyter-notebookipython-parallelmpi4py
bluerubez
bluerubez

Reputation: 300

Starting an IPython notebook MPI cluster

I am having a very hard time trying to start an IPython cluster that has MPI capabilities. First of all i have MPI and MPI4py installed and they are working. I finally figured out how to run an mpi python code from within ipython. However, I am trying to get a working mpi profile in IPython notebook and am at a loss. I have read all the documentation and even watched a lot of lectures on the subject and everytime i try to replicate what someone is doing i always get errors. Ok so if i type from the bash prompt:

ipython profile create --parallel --profile=mpi

I can go into the notebook and see that cluster. Then if i start it by hitting the start action button it starts up.But then if i do:

%pylab inline
from IPython.parallel import Client, error
cluster=Client(profile="mpi")
view=cluster[:]
view.block=True

%%px

from mpi4py import MPI
import numpy as np
import time

mpi=MPI.COMM_WORLD
bcast=mpi.bcast
barrier=mpi.barrier
rank=mpi.rank
print "MPI rank is %i/%i" % (rank,mpi.size)

I get:

MPI rank is 0/1
MPI rank is 0/1
MPI rank is 0/1
MPI rank is 0/1
MPI rank is 0/1
MPI rank is 0/1
MPI rank is 0/1
MPI rank is 0/1

Further if i do:

from Ipython import parallel

rc=parallel.Client()
rc.block=True
rc.ids

I get an error pointing at the rc=parallel.Client() line.So then both of these work fine with the default cluster... Also i tried 

ipcluster start -n 8 --engines=MPIEngineSetLauncher

From the bash prompt and i never get the prompt back anytime i try to start a cluster from there and if i put this from the notebook after i start profile=mpi with the GUI button:

from IPython.parallel import Client
c=Client(profile='mpi')
c.IPClusterEngines.engine_launcher_class='MPIEngineSetLauncher'

i get another error.I have been trying to do get this thing going so i can run a simulation for the last two days and am in a school where nobody knows how to do this. Please someone help me

Upvotes: 3

Views: 1808

Answers (1)

bluerubez
bluerubez

Reputation: 300

Okay. So i simply did not understand that after you run:

ipython profile create --parallel --profile=mpi

then:

ipcluster start -n 8 --engines=MPIEngineSetLauncher

You have to just open up a new terminal and let those engines run in the old terminal . THAT'S IT! Then you can go into ipython and all those commands work.

Upvotes: 1

Related Questions