CODEWITHSUNDEEP
mpi
Python_user
Python_user

Reputation: 1573

How does MPI_Send work when the application buffer size is greater than MPI buffer size?

How does MPI_Send() communicate the data to the receiving process if the size of the sending data is greater than the MPI buffer size? For example, let's say that I want to send 10 bytes of data (i.e., the size of my application buffer is 10B) in a single send message, but the MPI buffer has a fixed size of 6B. In this case, how does MPI_Send() send the data? Does it transfer first 6B and then transfer the remaining 4B? Or does it transfer only 6B?

Upvotes: 2

Views: 3530

Answers (1)

Wesley Bland
Wesley Bland

Reputation: 9082

There's a few different kinds of buffers involved in MPI messages so I want to make it clear what each of them do.

  1. Application Buffers - These buffers are allocated and managed by your application. Your data is stored in these, you do calculations with these, you pass these into MPI to tell it where to send or receive data. These are sized as large or larger than your data.

  2. Internal Buffers - These buffers are internal to MPI and may or may not even exist. There's nothing in the MPI Standard about these buffers or how they are supposed to act, how big they are supposed to be, etc. However, there are some reasonable assumptions that you can make.

    • Usually there will be some internal buffers that are used to speed up data transfers, especially for small messages. If your message is small enough, it could be copied into this buffer to be queued for transfer at a later time. This is what usually happens if you do a very small MPI_SEND. The call will return immediately, but the data may or may not have actually been sent to the receiving process. There are similar buffers on the receiving side so if a small message arrives before the application provides an application buffer where the data can be stored, it can be dropped into one of these smaller internal buffers until it's eventual destination is specified. This is usually called the eager protocol.
    • Sometimes, the internal buffers are either all used up or are too small for your message to be copied into them. In this case, MPI falls back to the rendezvous protocol. In this instance, MPI usually doesn't use an internal buffer at all, but retains control of the application buffer and sends data directly from there. If this is happening, your call to MPI_SEND won't return until the MPI library is done using the buffer and it's safe for your application to modify the data again.
  3. Special Buffers - There are other kinds of buffers that might provide special services, such as buffers on the network card that can speed up data transfers. The way these behave is usually specific to the type of network you're using.

Upvotes: 6

Related Questions