CODEWITHSUNDEEP
cparallel-processingopenmpi
Sterling
Sterling

Reputation: 4213

MPI_Gather with unequal amounts?

I am implementing matrix multiplication using MPI_Scatter and MPI_Gather. My code works fine if the number of processes divides evenly into the number of matrix rows,columns. However, it crashes on MPI_Gather when they do not divide evenly. This makes sense because MPI_Gather is expecting a certain amount from each process and will not receive that much from the last process. Is there a way to make this work? I looked into MPI_Gatherv, but I don't want spaces between the data - I just want it to receive less than the amount for the last process.

Currently, I am using MPI_Scatter to send each process the amount of:

ceil(N/size)

where size is the number of processes I am using and N is the number of rows and columns in the matrices. The last process will not receive as much data as the others. I am making sure to stop doing the multiplication when I reach the end of the data in the last process. I could change this scatter+multiplication part if there is no way to change my MPI_Gather call. Any help is appreciated.

Upvotes: 4

Views: 4220

Answers (3)

Youjun Hu
Youjun Hu

Reputation: 1314

MPI_Gather is used to gather information of the same size from various processes. If the size of arrays to be gathered is different among the processes, then we should use MPI_GatherV.

Upvotes: 2

firegurafiku
firegurafiku

Reputation: 3116

I think that wrapping MPI_Scatterv and MPI_Gatherv in helper functions is the way to go. Today I have written a pair of such wrappers as part of my summer school, feel free to use them if you still need a solution.

void fillCounts(int rank, int size, int totalCount, int *counts, int *offsets) {
    int i;
    int n;
    int sum;

    n = totalCount / size;
    for (i = 0; i < size; i++)
    counts[i] = n;

    for (i = 0; i < totalCount % size; i++)
        counts[i] += 1;

    sum = 0;
    for (i = 0; i < size; i++) {
    offsets[i] = sum;
    sum += counts[i];
    }
}

int scatter01(const void *sendBuf, int totalCount, MPI_Datatype dataType,
                    void *recvBuf, int *recvCount,
                    int root, MPI_Comm comm) {
    int rank;
    int size;
    int *counts;
    int *offsets;
    MPI_Comm_rank(comm, &rank);
    MPI_Comm_size(comm, &size);

    counts  = alloca(size * sizeof(int));
    offsets = alloca(size * sizeof(int));
    fillCounts(rank, size, totalCount, counts, offsets);

    *recvCount = counts[rank]; 
    return MPI_Scatterv(sendBuf, counts, offsets, dataType,
                        recvBuf, counts[rank, dataType, root, comm);
}

int gather01(void *recvBuf, int totalCount, MPI_Datatype dataType,
            const void *sendBuf, int root, MPI_Comm comm) {
    int rank;
    int size;
    int *counts;
    int *offsets;
    MPI_Comm_rank(comm, &rank);
    MPI_Comm_size(comm, &size);

    counts  = alloca(size * sizeof(int));
    offsets = alloca(size * sizeof(int));
    fillCounts(rank, size, totalCount, counts, offsets);

    return MPI_Gatherv(sendBuf, counts[rank], dataType,
               recvBuf, counts, offsets, dataType, root, comm);
}

Usage example:

double globData[N];
double partData[N/size+1];
int n;

scatter01(globData, N, MPI_DOUBLE, partData, &n, 0, MPI_COMM_WORLD);
gather01 (globData, N, MPI_DOUBLE, partData,     0, MPI_COMM_WORLD);

Upvotes: 2

Namila
Namila

Reputation: 754

Look at Figure 6 of below set of examples http://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node70.html

Upvotes: 0

Related Questions