CODEWITHSUNDEEP
mpi
coyotemk
coyotemk

Reputation: 143

getting the first proc arriving at some point, asynchronously

My program start with each process doing some work, and in the end one should gather the result and compute the final result. For now, everybody send the data to the proc 0, and it is the one that does the final computational task. I would like to find a to do this on the first process that finishes its local task.

Is there a trick on finding out this proc and tell to all the other procs asynchronously? I was thinking of using asynchronous collective operation but can't find a way to do it.

Thanks, Stole

Upvotes: 2

Views: 148

Answers (1)

David Daverio
David Daverio

Reputation: 333

As Gilles said, you can use a one sided operation (useful docs) to set a variable which will contain the rank of process which have finished first. You set the variable to -1, when a process have finished, it read that variable using one sided (first think about blocking the access of other processes to avoid races) if it is -1, the process set this variable to his rank and become the "master" if it is a number >=0 then the process become a slave (of that number).

(As gilles noted: With version >3.0 one can use MPI_Compare_and_swap to perform the Get and Put by once (which is probably faster that a Get followed by a Put))

This method works fine only if you have a real difference of execution time between processes and a small number of processes. So one should be careful when losing time on implementing such solution. If you have large load imbalance (and cant redesign the parallelization to increase load ballancing) that will make the job! Then if you targeting >10 processes, it is better to first group all processes on a nested subgroups grid. Then perform the trick at each level of groups (that avoid having 100k processes trying to read the same variable and wasting huge time!)

Here is a simple code which does it...

  #include <mpi.h>
  #include <thread>
  #include <chrono>
  #include <stdlib.h>
  #include <iostream>
  #include <time.h>


  using namespace std;

  int main(int argc, char** argv) {

      MPI_Init(NULL, NULL);

      int rank;
      MPI_Comm_rank(MPI_COMM_WORLD, &rank);

      srand (time(NULL)+rank);
      int wt = rand() % 5 + 1;
      cout <<"rank: "<< rank<< " sleep: "<< wt<<endl;



      //here is the magic...

      int * flag;
      MPI_Win window;
      int master;
      int compare = -1;
      MPI_Win_allocate(sizeof(int), sizeof(int),MPI_INFO_NULL,
                       MPI_COMM_WORLD, &flag, &window );


      *flag = -1;
      this_thread::sleep_for(chrono::seconds(wt));


      MPI_Win_lock (MPI_LOCK_EXCLUSIVE, 0, 0, window);
      MPI_Compare_and_swap(&rank, &compare, &master, MPI_INT, 0,0, window);
      MPI_Win_unlock (0, window);


      if(master == -1)
      {
        cout<< "rank: "<<rank<<". I am the master"<<endl;
      }
      else
      {
        cout<< "rank: "<<rank<<". I am the slave of "<<master<<endl;
      }


      MPI_Win_free( &window );
      MPI_Finalize();
  }

Outputs:

mpirun --oversubscribe  -np 4 ./test
rank: 2 sleep: 4
rank: 3 sleep: 1
rank: 0 sleep: 5
rank: 1 sleep: 2
rank: 3. I am the master
rank: 1. I am the slave of 3
rank: 2. I am the slave of 3
rank: 0. I am the slave of 3
mpirun --oversubscribe  -np 4 ./test
rank: 0 sleep: 2
rank: 1 sleep: 4
rank: 2 sleep: 1
rank: 3 sleep: 3
rank: 2. I am the master
rank: 0. I am the slave of 2
rank: 3. I am the slave of 2
rank: 1. I am the slave of 2

Upvotes: 1

Related Questions