CODEWITHSUNDEEP
c++mpi
Adrian
Adrian

Reputation: 1025

MPI - Send a std::vector<string> to another group of process

I am trying to send a vector to a group of process that are created using MPI_Comm_spawn.I have try everything on the internet but i con't solve it yet. Until now i manage to make this separate groups to communicate and the parend sends an int to the workers. I need to make it work with a std::vector

For parent

#include <stdio.h>
#include <string.h>
#include "mpi.h"

int main(int argc, char* argv[]) {
    int i;
    int my_rank; /* rank of process */
    int p; /* number of processes */

    /* start up MPI */
    MPI_Init(&argc, &argv);

    /* find out process rank */
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);

    /* find out number of processes */
    MPI_Comm_size(MPI_COMM_WORLD, &p);

    MPI_Group grp_world;
    MPI_Group grp_new;
    MPI_Comm newComm;

    MPI_Comm_group(MPI_COMM_WORLD, &grp_world); // get the group of processes for MPI_COMM_WORLD communicator

    MPI_Group_incl(grp_world, 1, &my_rank, &grp_new); // create a new group with only one member (my_rank process)

    MPI_Comm_create(MPI_COMM_WORLD, grp_new, &newComm); // create a new communicator for the new group

    int my_rank_grp; /* rank of process in grp_new */
    int p_grp; /* number of processes in grp_new */
    /* find out process rank in the new group */
    MPI_Comm_rank(newComm, &my_rank_grp);
    /* find out number of processes of the new group */
    MPI_Comm_size(newComm, &p_grp);

    printf("Process rank/size in WORLD/OWN GROUP: [%d/%d]/[%d/%d]\n",
            my_rank, p, my_rank_grp, p_grp);

    MPI_Comm workercomm; // intercommunicator
    int b = my_rank; // some data to be sent

    //[!!!] due to some MPI implementation problems, parallel calls the MPI_Comm_spawn trigger an error, so for the moment we just serialize these calls.
    for (i = 0; i < p; ++i) {
        if (i == my_rank) {
            MPI_Comm_spawn("/home/workspace_test/worker/Debug/worker",
                    MPI_ARGV_NULL, 3, MPI_INFO_NULL, 0, newComm, &workercomm,
                    MPI_ERRCODES_IGNORE );
        }
        MPI_Barrier(MPI_COMM_WORLD );
    }

    /* collective communication sample */
    MPI_Bcast(&b, 1, MPI_INT, MPI_ROOT, workercomm);
    MPI_Comm_free(&workercomm);

    /* shut down MPI */
    MPI_Finalize();

    return 0;
}

And after this i have a simple worker:

#include <stdio.h>
#include <string.h>
#include "mpi.h"

int main(int argc, char* argv[]) {
    int my_rank; /* rank of process */
    int p; /* number of processes */
    int p_remote; /* number of processes in parent group */

    /* start up MPI */
    MPI_Init(&argc, &argv);

    /* find out process rank */
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);

    /* find out number of processes */
    MPI_Comm_size(MPI_COMM_WORLD, &p);

    MPI_Comm parentcomm; // intercommunicator
    int b; // some data to be received

    MPI_Comm_get_parent(&parentcomm);

    /* find out number of processes in parent group */
    MPI_Comm_remote_size(parentcomm, &p_remote);


    MPI_Bcast(&b, 1, MPI_INT, 0, parentcomm);

    printf("[%d] Worker process: %d. Parent group size: %d!\n", b, my_rank, p_remote);

    MPI_Comm_free(&parentcomm);

    /* shut down MPI */
    MPI_Finalize();

    return 0;
}

All is good with this simple int Bcast but if i try to send a vector things dosen't work how i expect.

What i am trying to do?

I have a vector with paths (files from disk) and the number of paths will be divided to the number of process. Each process will receive a block of files and this process for each file will launch a worker proces that will make some verification on file.

What i try until now:

Maybe someone with some experience with mpi can give me some hints about how can i send the entire vector.

Thank you.

//edit

I have try to send data like this: On partent:

std::vector<std::string> m_image_data2;
m_image_data2.push_back("test");
m_image_data2.push_back("test");
m_image_data2.push_back("test");
m_image_data2.push_back("test");
m_image_data2.push_back("test");
char* data = const_cast<char*>(m_image_data2[0].c_str());
MPI_Bcast(data, 5, MPI_CHAR, MPI_ROOT, workercomm);
MPI_Comm_free(&workercomm);

And on the worker side:

/* find out number of processes in parent group */
MPI_Comm_remote_size(parentcomm, &p_remote);
std::vector<std::string> m_image_data2;
MPI_Bcast(&m_image_data2, 5, MPI_CHAR, 0, parentcomm);

So my problem right now appear when i try to send a vector of string between a spawned task.

Upvotes: 0

Views: 1215

Answers (1)

Shibli
Shibli

Reputation: 6149

Inspect the following code to see your mistakes.

master.cpp

#include <stdio.h>
#include <string>
#include <mpi.h>
#include <vector>
#include <cstring>

// Converts vector<string> to vector<char>.
std::vector<char> string_to_char(const std::vector<std::string>& strings) {
    std::vector<char> cstrings;
    cstrings.reserve(strings.size());
    for(std::string s: strings)
    {
        for(size_t i = 0; i < strlen(s.c_str()); ++i)
        {
            cstrings.push_back(s.c_str()[i]);
        }
    }

    return cstrings;
}

int main(int argc, char* argv[]) {
    ...
    std::vector<char> cstrings = string_to_char(m_image_data2);
    MPI_Bcast(cstrings.data(), cstrings.size(), MPI_CHAR, MPI_ROOT, workercomm);
    ...
}

worker.cpp

#include <stdio.h>
#include <string>
#include <mpi.h>
#include <vector>

int main(int argc, char* argv[]) {
    ...    
    std::vector<char> cstrings(20);
    MPI_Bcast(cstrings.data(), 20, MPI_CHAR, 0, parentcomm);

    // cstrings: testtesttesttesttest
    // print first four chars.
    printf("Worker process: %c%c%c%c\n", cstrings[0], cstrings[1], cstrings[2], cstrings[3]);
    ...
}

Upvotes: 1

Related Questions