Scatter a c++ vector of pairs with MPI

Question

I'm new to MPI so I'm asking your help. I need to scatter a vector of pairs in C++ with MPI. For this I created a MPI_Datatype "mytype". Then I use MPI_Scatterv because I don't want the scatter to depend nor on the size of the vector neither on the number of processes.

Here is my code:

#include<mpi.h>
#include<iostream>
#include<vector>

using std::cout;
using std::cerr;
using std::endl;

using std::vector;
using std::pair;

typedef vector<pair< int, int> > Vect;


void Build_mpi_type(unsigned int* i, unsigned int* j, MPI_Datatype* mytype)
{
    int array_of_blocklengths[2]={1,1};
    MPI_Datatype array_of_types[2]={MPI_INT, MPI_INT};
    MPI_Aint i_addr, j_addr;
    MPI_Aint array_of_displacements[2]={0};
    MPI_Get_address(i,&i_addr);
    MPI_Get_address(j,&j_addr);
    array_of_displacements[1]=j_addr-i_addr;
    MPI_Type_create_struct(2,array_of_blocklengths,array_of_displacements,array_of_types,mytype);
    MPI_Type_commit(mytype);
}


int main(int argc, char **argv)
{


    Vect esempio;

    esempio.push_back(std::make_pair(1,1));
    esempio.push_back(std::make_pair(2,2));
    esempio.push_back(std::make_pair(3,3));
    esempio.push_back(std::make_pair(4,4));

    unsigned int size=esempio.size();


    MPI_Init(&argc, &argv);

    int rank;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    int p;
    MPI_Comm_size(MPI_COMM_WORLD, &p);



    unsigned int* index_i=nullptr;
    unsigned int* index_j=nullptr;

    MPI_Datatype mytype;
    Build_mpi_type(index_i, index_j,&mytype);



    //cout<<esempio.size();
    vector<int> elem(p); //number of elements per process
    vector<int> disp(p); // first index for every local vector

    // how to distribute elements
    int split = size / p;
    int extra = size % p;
    for (int i = 0; i < extra; ++i) 
            elem[i] = split + 1;

    for (int i = extra; i < p; ++i) 
            elem[i] = split;

    for (int i = 1; i < p; ++i) 
            disp[i] = disp[i-1] + elem[i-1];

    int local_n = elem[rank]; 
    Vect local_v(local_n); 


    MPI_Scatterv(&esempio[0], &elem[0], &disp[0], mytype,
                 &local_v[0], local_n, mytype,
                 0, MPI_COMM_WORLD);

       if(rank==0){
               for(Vect::const_iterator pos = local_v.begin();pos != local_v.end(); ++pos)
cout<<pos->first<<" "<<pos->second<<endl;
    }

    MPI_Finalize();
    return 0;

}

Now the thing is that when I run the program with just one process (the root) I obtain this result:

1 1

2 2

0 0

0 0

When I run with two and three processes I have:

1 1

0 0

Finally with four processes:

1 0

Why is this happening? I really can't understand. At least with one process the result should be:

1 1

2 2

3 3

4 4

I hope someone of you who knows better MPI me can help me.

Answer 1

The trouble comes from arguments index_i and index_j given to Build_mpi_type(). This function will compute the offset between first and second element of pair<int,int>. Therefore, index_i should point to the first element and index_j to the second element of the same pair.

Something like this should do the trick :

 int* index_i=NULL;
 int* index_j=NULL;
 pair<int,int> bla;
 index_i=&(bla.first);
 index_j=&(bla.second);

You will need to change the prototype of function Build_mpi_type().

 void Build_mpi_type( int* i,  int* j, MPI_Datatype* mytype)

It may be better to suppress arguments i and j and create a dedicated function Build_mpi_type_pairint(MPI_Datatype* mytype)...

You may also add disp[0]=0; (even if it works without it...).

Bye,

Francis

Answer 2

If you can rely on boost libraries, take a look at Boost.MPI, I think it had support for serialization via boost.serialization and maybe pairs are supported.