CODEWITHSUNDEEP
cparallel-processingmpi
MsTais
MsTais

Reputation: 197

MPI_Gather junk in receiving buffer (MPI+C)

I am new to MPI and trying to parallelize the code on multiple threads. I need to communicate a big chunk of data back to the master thread, and can't clean up the memory garbage I get after MPI_Gather. Here is the sample code:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <mpi.h>

int main (int argc, char *argv[]) {

MPI_Init(&argc, &argv);
int rank, world_size;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &world_size);

double *total=NULL;
double parcial[15000];
int W=world_size*15000;

if (rank == 0) {
    total=malloc(sizeof(double) * world_size*15000);
}
else if (rank != 0) {
    for(int i=0; i<15000; i++)parcial[i] = rank*i*0.1;
}
MPI_Barrier(MPI_COMM_WORLD);

if(rank==0) for(int ii=0; ii<W; ii++)total[ii]=0.;

MPI_Gather(parcial,15000,MPI_DOUBLE,total,15000,MPI_DOUBLE,0,MPI_COMM_WORLD);

if (rank == 0) {
    int N=world_size*15000;
    for(int i=0; i<N; i++) printf("%f ", total[i]);
}

free(total);
MPI_Finalize();

}

If you run the code with multiple threads (I have tried 3,4,5...), it always has junk in the beginning of the receiving buffer even though I explicitly put total[ii] to zero before calling MPI_Gather and set a barrier. Is it because I physically have only two kernels? But I have read that MPI creates a virtual machine anyways. Is there a way to clean it and get a reliable gather?

Addition:

I figured, this junk is probably coming form zero-th thread. Why doesn't it go to zero after line 25?

Upvotes: 2

Views: 208

Answers (1)

ptb
ptb

Reputation: 2148

Excerpt from the MPI man page for MPI_Gather:

Each process (root process included) sends the contents of its send buffer to the root process.

Thus, the first 15000 elements of the recv buffer total will contain the elements of parcial on the root process. This was left unitialized in your code.

Edit: @Gilles commented that MPI_IN_PLACE could be used to avoid initialization on the root process. Here's an example showing its effect:

#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>


int main(int argc, char *argv[])
{
    int rank, size;

    MPI_Init(&argc, &argv);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    int *in_place_total = calloc(sizeof(int), size);
    int *total = calloc(sizeof(int), size);

    int sendval = rank-5000;

    MPI_Gather(&sendval, 1, MPI_INT,
           total, 1, MPI_INT, 0, MPI_COMM_WORLD);

    if (rank == 0)
        assert(total[0] == -5000);

    if (rank)
        MPI_Gather(&sendval, 1, MPI_INT,
           in_place_total, 1, MPI_INT, 0, MPI_COMM_WORLD);
    else
        MPI_Gather(MPI_IN_PLACE, 1, MPI_INT,
           in_place_total, 1, MPI_INT, 0, MPI_COMM_WORLD);

    if (rank == 0)
        assert(in_place_total[0] == 0);

    free(total);
    free(in_place_total);

    MPI_Finalize();

}

Upvotes: 3

Related Questions