CODEWITHSUNDEEP
mpi
Spiros
Spiros

Reputation: 2346

MPI datatype for structure of arrays

In an MPI application I have a distributed array of floats and two "parallel" arrays of integers: for each float value there are two associated integers that describe the corresponding value. For the sake of cache-efficiency I want to treat them as three different arrays, i.e. as a structure of arrays, rather than an array of structures.

Now, I have to gather all these values into the first node. I can do this in just one communication instruction, by defining an MPI type, corresponding to a structure, with one float and two integers. But this would force me to use the array of structures pattern instead of the structure of arrays one.

So, I can choose between:

Do you know a third option that would allow me do have the best of both worlds, i.e. having a single communication and keeping the cache-efficient configuration?

Upvotes: 3

Views: 1996

Answers (2)

Solkar
Solkar

Reputation: 1226

E.g. by using the *array_of_blocklength* parameter of MPI_Type_create_struct

// @file mpi_compound.cpp
#include <iterator>
#include <cstdlib> // for rng
#include <ctime>   // for rng inits
#include <iostream>
#include <algorithm>

#include <mpi.h>

const std::size_t N = 10; 

struct Asset {
    float f[N];
    int m[N], n[N];

    void randomize() {
        srand(time(NULL));
        srand48(time(NULL));
        std::generate(&f[0], &f[0] + N, drand48);
        std::generate(&n[0], &n[0] + N, rand);
        std::generate(&m[0], &m[0] + N, rand);
    }
};


int main(int argc, char* argv[]) {

    MPI_Init(&argc,&argv);
    int rank,comm_size;
    MPI_Status stat;

    MPI_Comm_rank(MPI_COMM_WORLD,&rank);
    MPI_Comm_size(MPI_COMM_WORLD,&comm_size);

    Asset a;

    MPI_Datatype types[3] = { MPI_FLOAT, MPI_INT, MPI_INT };
    int bls[3] = { N, N, N }; 
    MPI_Aint disps[3];
    disps[0] = 0;
    disps[1] = int(&(a.m[0]) - (int*)&a)*sizeof(int);
    disps[2] = int(&(a.n[0]) - (int*)&a)*sizeof(int);

    MPI_Datatype MPI_USER_ASSET;

    MPI_Type_create_struct(3, bls, disps, types, &MPI_USER_ASSET);
    MPI_Type_commit(&MPI_USER_ASSET);

    if(rank==0)  {
        a.randomize();
        std::copy(&a.f[0], &a.f[0] + N, std::ostream_iterator<float>(std::cout, " "));
        std::cout << std::endl;
        std::copy(&a.m[0], &a.m[0] + N, std::ostream_iterator<int>(std::cout, " "));
        std::cout << std::endl;
        std::copy(&a.n[0], &a.n[0] + N, std::ostream_iterator<int>(std::cout, " "));
        std::cout << std::endl;
        MPI_Send(&a.f[0],1,MPI_USER_ASSET,1,0,MPI_COMM_WORLD);
    } else {
        MPI_Recv(&a.f[0],1,MPI_USER_ASSET,0,0,MPI_COMM_WORLD, &stat);
        std::cout << "\t=> ";
        std::copy(&a.f[0], &a.f[0] + N, std::ostream_iterator<float>(std::cout, " "));
        std::cout << std::endl << "\t=> ";
        std::copy(&a.m[0], &a.m[0] + N, std::ostream_iterator<int>(std::cout, " "));
        std::cout << std::endl << "\t=> ";
        std::copy(&a.n[0], &a.n[0] + N, std::ostream_iterator<int>(std::cout, " "));
        std::cout << std::endl;
    }

    MPI_Type_free(&MPI_USER_ASSET);

    MPI_Finalize();
    return 0;
}

worked with

mpirun -n 2 ./mpi_compound

with mpich2 v1.5 (HYDRA) on x86_86 linux and g++ 4.4.5-8 - based mpic++

Upvotes: 0

Madsen
Madsen

Reputation: 406

You take take a look at Packing and Unpacking.

http://www.mpi-forum.org/docs/mpi-11-html/node62.html

However, I think if you want to pass a same "structure" often you should define you own MPI derivate type.

Upvotes: 0

Related Questions