CODEWITHSUNDEEP
mpiscalability
gspr
gspr

Reputation: 11227

Is `MPI_COMM_WORLD` a bad idea if only pairs of ranks ever communicate?

I have very little experience with MPI, so please forgive the naïveté of this question.

I have what I thought was a relatively simple MPI program: a large amount of independent tasks need to be computed, and assembled into a certain array. I do this by letting all ranks talk to rank 0, requesting new entries to compute, and reporting the result back to rank 0. Each communication is of the order of just hundreds of bytes, and always happens between rank 0 and any one of the other ranks. Is it a bad idea to keep all the ranks in the MPI_COMM_WORLD communicator in such a setup? Should I split into a bunch of separate communicators consisting or rank 0 & 1, rank 0 & 2, rank 0 & 3, … , rank 0 & n? Or is it OK to stay in MPI_COMM_WORLD as long as all communications just go to one other rank anyway?

Upvotes: 1

Views: 745

Answers (1)

j23
j23

Reputation: 3530

Communicators are heavy-weight objects. Creating a new communicator takes time and consumes internal MPI resources.

It is better to create new communicators for the following cases:

  • When different abstraction layers within your application need “safe” communication scopes.
  • When you need to subset the processes from a parent communicator (e.g., you have a significant operation that only needs to be performed by half the processes in MPI_COMM_WORLD).
  • When you need to re-order the processes from a parent communicator (e.g. your MPI processes have a logical ordering that is different than their “native” MPI_COMM_WORLD rank).

In short: Only create new communicators when you need a whole new/safe communication scope, or you need to change your existing scope (e.g., subset and/or reorder the member processes).

Reference: Here

Upvotes: 3

Related Questions