CODEWITHSUNDEEP
mpi
Stefan Bogdan
Stefan Bogdan

Reputation: 21

MPI Receive from many proceses

Here is my code:

 if (rank != 0) {

        // trimitem numarul de pixeli prelucrati
        rc = MPI_Send(&pixeli, 1, MPI_INT, 0, tag, MPI_COMM_WORLD);

        // trimitem coordonatele de unde am inceput prelucrarea 
        rc = MPI_Send(&first_line, 1, MPI_INT, 0, tag, MPI_COMM_WORLD);
        rc = MPI_Send(&first_col, 1, MPI_INT, 0, tag, MPI_COMM_WORLD);

        for (i = 0; i < pixeli; i++) {
            rc = MPI_Send(&results[i], 1, MPI_INT, 0, tag, MPI_COMM_WORLD);         
        }       
}
else {

        for (i = 1; i < numtasks; i++) {
            rc = MPI_Recv(&received_pixels, 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);
            results_recv = (int*) calloc (received_pixels, sizeof(int));

            rc = MPI_Recv(&start_line_recv, 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);
            rc = MPI_Recv(&start_col_recv, 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);

            for (j = 0; j < received_pixels; j++) {
                rc = MPI_Recv(&results_recv[j], 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);                 
            }
            free(results_recv);
}

If I run this with 2 proceses it is ok because one will send and the other one will receive.

If I run this with 4 proceses I receive the following error messages:

Fatal error in MPI_Recv: Other MPI error, error stack:
MPI_Recv(186)...........................: MPI_Recv(buf=0xbff05324, count=1, MPI_INT, src=1, tag=1, MPI_COMM_WORLD, status=0xbff053ec) failed
MPIDI_CH3I_Progress(461)................: 
MPID_nem_handle_pkt(636)................: 
MPIDI_CH3_PktHandler_EagerShortSend(308): Failed to allocate memory for an unexpected message. 261895 unexpected messages queued.

What should I do to fix this?

Upvotes: 1

Views: 1355

Answers (2)

Aaron Altman
Aaron Altman

Reputation: 1755

These lines:

for (i = 0; i < pixeli; i++) {
    rc = MPI_Send(&results[i], 1, MPI_INT, 0, tag, MPI_COMM_WORLD);         
}

and the corresponding MPI_Recvs look like they're essentially reimplementing MPI_Gather. Using the MPI_Gather call with size set to pixeli instead of 1 may allow the implementation to schedule the sends and receives more efficiently, but more importantly, it will probably drastically cut down on the total number of send/receive pairs needed to complete the whole batch of communication. You could do similar by removing the for loop and doing:

rc = MPI_Send(&results[i], pixeli, MPI_INT, 0, tag, MPI_COMM_WORLD);

but again, using the builtin MPI_Gather would be the preferred way of doing it.

Upvotes: 1

francis
francis

Reputation: 9817

The shortest answer is to tell you to use synchronious communications, that is MPI_Ssend() instead of MPI_Send().

The trouble is that you send to many messages which are buffered (i guess...but i though MPI_Send() was blocking...). The memory consumption goes up until failure...Synchronious messages avoid buffering but it does not reduce the number of messages and it may be slower.

You can reduce the number of messages and increase performances by sending many pixels at once : second argument of MPI_Send() or MPI_Recv()... Sending a buffer of 3 int [pixeli,first_line,first_col] would also limit communications.

#include <stdio.h>
#include <time.h>
#include <stdlib.h>
#include <string.h>
#include "mpi.h"

int main(int argc,char *argv[])
{
    int rank, size;


    MPI_Init(&argc, &argv);
    MPI_Comm_size(MPI_COMM_WORLD, &size);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);

    int pixeli=1000000;
    int received_pixels;
    int first_line,first_col,start_line_recv,start_col_recv;
    int tag=0;
    int results[pixeli];
    int i,j;
    for(i=0;i<pixeli;i++){
        results[i]=rank*pixeli+i;
    }
    int* results_recv;
    int rc;
    MPI_Status Stat;
    if (rank != 0) {

        // trimitem numarul de pixeli prelucrati
        rc = MPI_Ssend(&pixeli, 1, MPI_INT, 0, tag, MPI_COMM_WORLD);

        // trimitem coordonatele de unde am inceput prelucrarea 
        rc = MPI_Ssend(&first_line, 1, MPI_INT, 0, tag, MPI_COMM_WORLD);
        rc = MPI_Ssend(&first_col, 1, MPI_INT, 0, tag, MPI_COMM_WORLD);

        MPI_Send(&results[0], pixeli, MPI_INT, 0, tag, MPI_COMM_WORLD); 
        //for (i = 0; i < pixeli; i++) {
        //    rc = MPI_Send(&results[i], 1, MPI_INT, 0, tag, MPI_COMM_WORLD);         
        //}       
    }
    else {

        for (i = 1; i < size; i++) {
            rc = MPI_Recv(&received_pixels, 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);
            results_recv = (int*) calloc (received_pixels, sizeof(int));

            rc = MPI_Recv(&start_line_recv, 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);
            rc = MPI_Recv(&start_col_recv, 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat);

            MPI_Recv(&results_recv[0], received_pixels, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat); 

            //for (j = 0; j < received_pixels; j++) {
            //    rc = MPI_Recv(&results_recv[j], 1, MPI_INT, i, tag, MPI_COMM_WORLD, &Stat); 
            //printf("proc %d %d\n",rank,results_recv[j]);                
            //}
            free(results_recv);
        }
    }


    MPI_Finalize();
    return 0;
}

Bye,

Francis

Upvotes: 0

Related Questions