CODEWITHSUNDEEP
hadoop
Pradeep Rai
Pradeep Rai

Reputation: 11

Number of reducer in map reduce

I have set 3 reducer in driver class in MR.

it means three partitions will be created..

but mapper is emiting only two unique key say male and female.

In this case, how times Reducer and reduce functions will run?

Upvotes: 1

Views: 151

Answers (2)

Binary Nerd
Binary Nerd

Reputation: 13927

If you set 3 reducers they will all run. However, if you only have two keys at most two of the reducers will actually receive data to process.

Upvotes: 1

Dennis Jaheruddin
Dennis Jaheruddin

Reputation: 21563

Your question is a bit vague, but I could come up with two explanations of what may happen:

1. Two stage reduction

  • Reducer 1 reduces all male results
  • Reducer 2 reduces all female results
  • Resucer 3 reduces the output of Reducer 1 and 2.

2. Splitting a job

  • Reducer 1 reduces all male results
  • Reducer 2 reduces the first half of female results
  • Reducer 3 reduces the second half of female results

Upvotes: 1

Related Questions