CODEWITHSUNDEEP
cparallel-processingmpi
user2662886
user2662886

Reputation: 21

MPI Gather From SELECT Processors Only

I'm looking for a function in MPI similar to allgather except it gathers data from only a certain subset of all the processors.

Furthermore, I would like to combine the data by an order that does not depend on the rank of the processor.

Is there a function like this? If not, how could implement this efficiently?

Thank you!

Upvotes: 2

Views: 3185

Answers (2)

Spiros
Spiros

Reputation: 2356

Basically there are two solutions to your problem. The first solution, is to create a communicator, where you place the nodes that have to send data; however, this does not allows you to shuffle the gathered data. The second solution is to use MPI_Gatherv instead of MPI_Gather.

MPI_Gatherv does the same as MPI_Gather, but you specify how many elements are you gathering from each processor and where you want to place them. Consider e.g. that you may want to gather 2 values of process 0, 3 values of process 1, no values from process 2 and 1 values from process 3.

Then you have to call MPI_Gatherv with the following parameters (I put arrays in square brackets, so this is not valid code; mycount is the number of elements that the current process has to send, so it will be 2 for process 0, 3 for process 1,...):

MPI_Gatherv(sendbuffer, mycount, type,
    recvbuffer, [2, 3, 0, 1], [0, 2, 5, 5], type,
    root, comm)

Notice that the second array, for the parameter "displs" specifies the offset within the recvbuffer where the elements received from the corresponding processes will be saved:

ProcID    Portion    Size
  0       [0, 2)      2
  1       [2, 5)      3
  2       [5, 5)      0
  3       [5, 6)      1

If you prefer e.g. to store the values in process-reversed order, in the sense that the values coming from process 3 are at the beginning and thos from the process 0 at the end, you should do instead:

MPI_Gatherv(sendbuffer, mycount, type,
    recvbuffer, [2, 3, 0, 1], [4, 1, 1, 0], type,
    root, comm)

This way the "occupational plan" looks like:

ProcID    Portion    Size
  0       [4, 6)      2
  1       [1, 4)      3
  2       [1, 1)      0
  3       [0, 1)      1

If you want a random distribution, you will "just" have to randomly generate an array "displ", paying (a lot of) attention to make it consistent.

As two final remarks, notice that the two arrays recvcounts and displs are only relevant to root; on the other nodes it is safe to pass the null-pointer. The other remark is that you have to call MPI_Gatherv also in the nodes that do not send data: in these cases mycount has to be set to 0; if you want to avoid to call MPI_Gatherv on the nodes that have no values to send, you have to set-up a differenc communicator and then call MPI_Gatherv.

Upvotes: 2

Wesley Bland
Wesley Bland

Reputation: 9072

I don't think there's any way to change the order inside the MPI call itself (though you can easily reorder things yourself afterward if you like), however solving the first problem is relatively easy.

In MPI, it's not required that all of the collective calls happen on MPI_COMM_WORLD (which contains all of the processes started at the beginning of the execution). You can create new communicators that contain whatever subset of the original group of processes that you like. The call to do this is MPI_COMM_CREATE.

To use MPI_COMM_CREATE, you need to manipulate the MPI_Group of processes that you want to put in the new communicator using the appropriate functions (MPI_GROUP_EXCL, MPI_GROUP_INCL, MPI_GROUP_INTERSECTION, etc.). There's usually plenty of tutorials on the web where you can find out how to do these sorts of things if it's not immediately obvious.

Upvotes: 1

Related Questions