CODEWITHSUNDEEP
cparallel-processingmpi
dmitru
dmitru

Reputation: 371

Why does MPI_Sendrecv block?

Suppose I have n processes and each of them possesses a piece of local data, say, an int.

Now I want to re-implement MPI_Allreduce() function, that is calculate the global sum of all those ints and then broadcast the resulting sum back to the processes.

I tried to do that like in the code below:

  int temp;
  int my_sum = temp = my_data;
  for (int i = 1; i < size; ++i) {
    int partner = (my_rank + 1) % size;
    MPI_Sendrecv_replace(&temp, 1, MPI_INT, 
                         partner, 0, 
                         my_rank, 0, 
                         MPI_COMM_WORLD, MPI_STATUS_IGNORE);
    my_sum += temp;
  }

so that processes communicate in a ring fahion, but it's blocking. Why? How to change the code so that it works properly?

NOTE: Please, don't suggest alternative (better) solutions to the problem (clearly, one of them is to use MPI_Allreduce() function). My goal is to understand why this code snippet doesn't work as I think it should.

Upvotes: 4

Views: 2196

Answers (1)

Jonathan Dursi
Jonathan Dursi

Reputation: 50927

Everyone is trying to receive from themeselves, but they are sending to partner; since partner will never receive the message being sent, and my_rank will never get a message from itself here, there's a hang.

If you're trying send data to parter, then you need to make sure partner is receiving from you, which means everyone needs to be receiving from (myrank - 1 + size) % size:

int spartner = (my_rank + 1) % size;
int rpartner = (my_rank - 1 + size) % size;
MPI_Sendrecv_replace(&temp, 1, MPI_INT,
                     spartner, i,
                     rpartner, i,
                     MPI_COMM_WORLD, MPI_STATUS_IGNORE);

That way, rank 3 (say) is sending to 4 and rank 4 is receiving from rank 3 at each iteration, and so the sendrecv completes. (I've also taken the liberty of giving each iteration through the loop its own tag, which isn't really necessary here but often helps find bugs if there are other mismatched-message type errors.)

Upvotes: 7

Related Questions