CODEWITHSUNDEEP
mpi
WhatsupAndThanks
WhatsupAndThanks

Reputation: 45

mpi_alltoall on nonequal number of grids and processes

I understand the general usage of MPI_alltoall, which can be described by the following figureenter image description here

But in practice, it is almost not always that the number of processes will equal to the number grids. The above case assume process = grid = 4. If numbers are not equal, I will have rectangular grids. Below I show an example showing a similar alltoall operation, but nonequal number of grids and processes (grid = 8, process = 2). enter image description here

My question is then very straightforward, how should I achieve that? I have looked over alltoallv, but I don't think it will work. Any suggestions are welcome. Thank you

Upvotes: 1

Views: 559

Answers (1)

Gilles Gouaillardet
Gilles Gouaillardet

Reputation: 8395

a "natural" alltoall would be

MPI_Alltoall(sbuf, 4, MPI_INT, rbuf, 4, MPI_INT, MPI_COMM_WORLD);

and you would end up with

P0 = { A0, A1, A2, A3, C0, C1, C2, C3}
P1 = { B0, B1, B2, B3, D0, D1, D2, D3}

your case is a bit convoluted and you have to use (complex) derived datatypes. (note I did not free the intermediate datatypes in order to keep the code readable)

    MPI_Datatype tmp, stype, rtype;

    /* derived datatype for send */
    MPI_Type_vector(2, 1, 4, MPI_INT, &tmp); /* {0, 4} */
    MPI_Type_create_resized(tmp, 0, 4, &tmp); /* next type starts at 1 */
    MPI_Type_contiguous(2, tmp, &tmp); /* {0, 4, 1, 5} */
    MPI_Type_create_resized(tmp, 0, 8, &stype); /* next type starts at 2, likely unnecessary */
    MPI_Type_commit(&stype);

    /* derived datatype for recv */
    MPI_Type_vector(2, 2, 4, MPI_INT, &tmp); /* {0, 1, 4, 5 } */
    MPI_Type_create_resized(tmp, 0, 8, &rtype); /* next type starts at 2 */
    MPI_Type_commit(&rtype);

    /* all2all */
    /* thanks to the derived datatypes :
       P0 sends {A0, B0, A1, B1} to P0 and {A2, B2, A3, B3} to P1
       P0 receives {A0, B0, .., .., A1, B1, .., ..} from itself, and
       { .., .., C0, D0, .., .., C1, D1} from P1 } */
    MPI_Alltoall(sbuf, 1, stype, rbuf, 1, rtype, MPI_COMM_WORLD);

Upvotes: 1

Related Questions