CODEWITHSUNDEEP
mpi
arbitUser1401
arbitUser1401

Reputation: 575

Cost of MPI_reduce versus MPI_allreduce

What is the cost of MPI_Reduce and MPI_allreduce (in terms of network latency and bandwidth).

I'd appreciate if you can also provide reference to algorithm. Do the cost model changes with message size?

Upvotes: 2

Views: 2532

Answers (2)

Jeff Hammond
Jeff Hammond

Reputation: 5662

These papers describe MPI reduction algorithms:

In general, there are many different protocols used for MPI collections, with selection based upon message size, how many processes are involved, etc.

In the case of MPI reductions, some form of reduction tree is instantiated on the MPI communicator. For large messages, Rabenseifner's algorithm may be more efficient.

Upvotes: 1

JamesTullos
JamesTullos

Reputation: 242

I would recommend downloading a benchmark to test this. The Intel® MPI Benchmarks can be run only using MPI_Reduce and MPI_Allreduce to show expected times for each. This is open source, and you can compile it with your preferred MPI implementation and test with various parameters. By default, you'll get a set of results spanning message sizes up to 4 MB. You can download it at https://software.intel.com/en-us/articles/intel-mpi-benchmarks.

Upvotes: 0

Related Questions