CODEWITHSUNDEEP
hadoopmapreduce
Haozhun
Haozhun

Reputation: 6511

Does combiner work on results from multiple mappers?

If multiple mappers are executed on the same node, will combiner combine the results from multiple mappers?

I can't find the answer for this in documents or books. And combiner examples I can find all seem to make a difference even if it can aggregate results from one mapper only.

Upvotes: 2

Views: 1052

Answers (2)

Haozhun
Haozhun

Reputation: 6511

The wording in the Yahoo's tutorial (mentioned in Leonard's answer) seems to be misleading.

As of now, we get a combiner for every map task. The functionality suggested by this question is under development still: https://issues.apache.org/jira/browse/MAPREDUCE-4502

Upvotes: 2

LeonardBlunderbuss
LeonardBlunderbuss

Reputation: 1274

From Yahoo's Hadoop Tutorial:

The Combiner will receive as input all data emitted by the Mapper instances on a given node. The output from the Combiner is then sent to the Reducers, instead of the output from the Mappers. The Combiner is a "mini-reduce" process which operates only on data generated by one machine.

So to answer your question, yes.

Upvotes: 4

Related Questions