Question in MPI_Gather parallel programming

Question

I'm a newbie that inserting in parallel programing. For my understanding I try to code by myself. Then i found that, I'm not understand in MPI_Gather. Let see the code first then I will explain later.

#include "mpi.h"
#include <stdio.h>
int main (int argc, char *argv[]) {
int size;
int rank;
int a[12];
int i;
int start,end;

MPI_Init(&argc, &argv);

MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
if(rank==0)
{
    for(i=0;i<12;i++)
    {
        a[i] = 100;
    }
}

start = 12/size*rank;
end = 12/size*(rank+1);
for(i=start;i<end;i++)
{
    a[i] = rank;
    printf("rank %d set a[%d] equal to %d\n",rank,i,rank);
}
MPI_Gather(&a[start],12/size*rank,MPI_INT,a,12/size*rank,MPI_INT,0,MPI_COMM_WORLD);
if(rank==0)
{
    for(i=0;i<12;i++)
    {
        printf("%d    %d\n",i,a[i]);
    }
}
MPI_Finalize();
return 0;
}

For this code my aim is to gather the value in the sub array that generate in each process and keep it in array a. Then let process 0 to print it.

First of all, I initial all value in array to 100

Then, I calculate the starting index and end index for each process(in this case number of process much be a multiple of 12)

Next, I assign the value to array that equal to it rank

Next, I gather it

Finally, I print it on the screen

Here is a out put for 3 process of size = 3

rank 2 set a[8] equal to 2

rank 2 set a[9] equal to 2

rank 2 set a[10] equal to 2

rank 2 set a[11] equal to 2

rank 1 set a[4] equal to 1

rank 1 set a[5] equal to 1

rank 1 set a[6] equal to 1

rank 1 set a[7] equal to 1

rank 0 set a[0] equal to 0

rank 0 set a[1] equal to 0

rank 0 set a[2] equal to 0

rank 0 set a[3] equal to 0

0 0

1 0

2 0

3 0

4 100

5 100

6 100

7 100

8 100

9 100

10 100

11 100

As you see, The gather collect only the first length of data, how can i fix it.

Thank you in advance.

Answer 1

Plz, try that code!

#include "mpi.h"
#include <stdio.h>
int main (int argc, char *argv[]) {
int size;
int rank;
int a[12] , b[12];
int i;
int start,end;

MPI_Init(&argc, &argv);

MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);
if(rank==0)
{
    for(i=0;i<12;i++)
    {
        a[i] = 100;
    }
}

start = 12/size*rank;
end = 12/size*(rank+1);
for(i=start;i<end;i++)
{
    a[i] = rank;
    printf("rank %d set a[%d] equal to %d\n",rank,i,rank);
}
MPI_Gather(&a[start],12/size,MPI_INT,b,12/size,MPI_INT,0,MPI_COMM_WORLD);
if(rank==0)
{
    for(i=0;i<12;i++)
    {
        printf("%d    %d\n",i,b[i]);
    }
}
MPI_Finalize();
return 0;
}

Answer 2

Congratulations, you almost got it on your first try.

You need to change your MPI_Gather line to

MPI_Gather(&a[start],12/size,MPI_INT,a,12/size,MPI_INT,0,MPI_COMM_WORLD);

The signature for MPI_Gather is

#include "mpi.h"
int MPI_Gather ( void *sendbuf, int sendcnt, MPI_Datatype sendtype, 
                void *recvbuf, int recvcount, MPI_Datatype recvtype, 
                int root, MPI_Comm comm )

So you had sendbuf, sendtype, recvbuf, recvtype, root, and comm correct; the problem were the counts. You want to send not 12/size*rank, which is start, integers; you want to send 12/size, or just end-start integers, and receive that many from each process as well.

MPI_Gather requires that every process is sending the same amount of information (if not, you want to use MPI_Gatherv() instead). In your case, the first process was being told to send 0, the second to send 4, and the third to send 8; the standard doesn't specify what to do in that case, and it looks like in the end, nothing got sent. (Or certainly not received; your root process, 0, was told to send 0 ints, so it presumably expected 0 intgs from every other task). So what you were left with was just what rank 0 would have had even before the gather; 100 everywhere except for its own values, which were 0.

By correcting the gather, you should have the right results:

rank 0 set a[0] equal to 0
rank 0 set a[1] equal to 0
rank 0 set a[2] equal to 0
rank 0 set a[3] equal to 0
rank 1 set a[4] equal to 1
rank 1 set a[5] equal to 1
rank 1 set a[6] equal to 1
rank 1 set a[7] equal to 1
rank 2 set a[8] equal to 2
rank 2 set a[9] equal to 2
rank 2 set a[10] equal to 2
rank 2 set a[11] equal to 2
0    0
1    0
2    0
3    0
4    1
5    1
6    1
7    1
8    2
9    2
10    2
11    2