Need some clarification on MPI

Question

I'm looking into various parallel programming models and I just stumbled upon MPI. There are a few things I'm not sure about. From what I understand:

1. the main function of MPI is to allow communication between related processes running in parallel in a cluster. Is this correct?
2. programs are run on the master node, which in turn communicates specific tasks to the worker nodes - and receives the results. There is no need for a copy of the code to be on each worker node.

Also, the few updated MPI frameworks I found only seem to support a restricted set of languages. I was looking at experimenting with this in Java but all the Java MPI libraries I found were either dated or didn't seem to have the maturity of those for C and Fortran. Why is that? I think that message passing would be a pretty useful feature for anyone who writes a parallel or distributed program, so why isn't MPI more popular outside of its niche?

Answer 1

MPI is traditionally a SPMD model where one code runs on lots of processes (usually distributed over a number of nodes/processors/cores/etc.). Those processes explicitly pass messages using the calls in MPI (Message Passing Interface). Then the individual processes can go off to do their work and communicate again when necessary. It's not a task language where the jobs can be automatically distributed to the worker nodes. There are other languages that do that sort of thing (Hadoop/MapReduce, Charm++, etc. might fit more into that model).

In the MPI Standard (http://www.mpi-forum.org/docs/docs.html) that's decided on by the MPI Forum, there are specific language requirements that are specified. Up to now, there have only been three languages that are required for an MPI implementation: C, C++, and FORTRAN (and actually the C++ requirement has been removed). The reason for this is that these are the languages most commonly used by the scientific applications that use MPI. This doesn't prevent other language bindings from existing, but supporting every language is a big burden for implementations and not always feasible. There are lots of other language bindings available:

• Python: http://mpi4py.scipy.org
• Java: http://www.open-mpi.org/faq/?category=java
• Perl: http://search.cpan.org/~josh/Parallel-MPI/MPI.pm
• C++: http://www.boost.org/doc/libs/1_54_0/doc/html/mpi.html

I'm sure there are more than this out there for some more exotic languages. Just search for MPI and you're sure to find information. MPI is actually pretty popular, even outside of the scientific community. There have been attempts to use it in the video game community, distributed databases, and others. However, scientific computing continues to be one of the biggest users of MPI and remains one of the Forum's main focuses.