CODEWITHSUNDEEP
mpi
armando
armando

Reputation: 1480

MPI overlapping communication and computation

my current understanding of MPI nonblocking routines is that they allow for the overlapping of communication and computation. However, I also understood that this overlapping is not guaranteed by the MPI implementation. Then, what could be the factors that inhibit the overlapping? Thanks.

Upvotes: 0

Views: 425

Answers (1)

Victor Eijkhout
Victor Eijkhout

Reputation: 5794

Non-blocking routines were not primarily motivated by latency hiding (I'll use this as a shorter synonym of "overlap of computations and communication"): the prime use was to be able to write deadlock/serialization-free code. For the longest time, achieving actual performance improvement required periodically activating the MPI library by MPI_Iprobe or such tricks. The basic problem was that during your computation, there was no guarantee that the MPI layer would do anything at all.

The problem of forcing "MPI progress" still persists, but these days MPI implementations such as from Intel or mvapich (sorry, I don't know about OpenMPI) have environment variables with which you can force "progress threads". Also, network cards may be clever enough to work while your processor is otherwise engaged. And even with all this, improvement is not guaranteed because of the overhead you are introducing.

Upvotes: 1

Related Questions