CODEWITHSUNDEEP
c++arraysalgorithmparallel-processingmpi
0x56794E
0x56794E

Reputation: 21281

C++ split integer array into chunks

I guess my question has 2 parts:

(1) Is this the right approach to send different chunks of an array to different processors? Let's say I have n processors whose rank ranges from 0 to n-1. I have an array of size d. I want to split this array into k equally-sized chunks. Assume d is divisible by k. I want to send each of these chunks to a processor whose rank is less than k. It would be easy if I can use something like MPI_Scatter, but this function sends to EVERY OTHER processors, and I only want to send to a certain number of procs. So what I did was that I have a loop of k iterations and do k MPI_Isend's. Is this efficient?

(2) If it is, how do I split an array into chunks? There's always the easy way, which is

int size = d/k;
int buffs[k][size];

for (int rank = 0; rank < k; ++rank)
{
    for (int i = 0; i < size ++i)
      buffs[rank][i] = input[rank*size + i];
    MPI_Isend(&buffs[rank], size, MPI_INT, rank, 1, comm, &request);
}

Upvotes: 0

Views: 1506

Answers (1)

Hristo Iliev
Hristo Iliev

Reputation: 74355

What you are looking for is MPI_Scatterv which allows you to explicitly specify the length of each chunk and its position relative to the beginning of the buffer. If you don't want to send data to certain ranks, simply set the length of their chunks to 0:

int blen[n];
MPI_Aint displ[n];

for (int rank = 0; rank < n; rank++)
{
    blen[rank] = (rank < k) ? size : 0;
    displ[rank] = rank * size;
}

int myrank;
MPI_Comm_rank(MPI_COMM_WORLD, &myrank);

MPI_Scatterv(input, blen, displ, MPI_INT,
             mybuf, myrank < k ? size : 0, MPI_INT,
             0, MPI_COMM_WORLD);

Note that for rank >= k the displacements will run past the end of the buffer. That is all right since block lengths are set to zero for rank >= k and no data will be accessed.

As for your original approach, it is not portable and might not always work. The reason is that you are overwriting the same request handle and you never wait for the sends to complete. The correct implementation is:

MPI_Request request[k];

for (int rank = 0; rank < k; ++rank)
{
    MPI_Isend(&input[rank*size], size, MPI_INT, rank, 1, comm, &request[rank]);
}
MPI_Waitall(k, request, MPI_STATUSES_IGNORE);

The most optimal implementation would be to use MPI_Scatter in a subcommunicator:

MPI_Comm subcomm;
MPI_Comm_split(MPI_COMM_WORLD, myrank < k ? 0 : MPI_UNDEFINED, myrank,
               &subcomm);
// Now there are k ranks in subcomm
// Perform the scatter in the subcommunicator
if (subcomm != MPI_COMM_NULL)
   MPI_Scatter(input, size, MPI_INT, mybuf, size, MPI_INT, 0, subcomm);

The MPI_Comm_split call splits MPI_COMM_WORLD and creates a new communicator from all original ranks less than k. It uses the original rank as key for ordering the ranks in the new communicator, therefore rank 0 in MPI_COMM_WORLD becomes rank 0 in subcomm. Since MPI_Scatter often performs better than MPI_Scatterv, this one is the most optimal solution.

Upvotes: 1

Related Questions