CODEWITHSUNDEEP
hadoopmapreduce
Yuhao
Yuhao

Reputation: 1610

Why cannot Reducer.class used as a real reducer in Hadoop MapReduce?

I noticed that Mapper.class can be used as a real mapper in a phase, together with a user-defined reducer. For example,

Phase 1:
Mapper.class ->  WordCountReduce.class

This will work. However, Reducer.class cannot be used the same way. Namely something like

Phase 2:
WordReadMap.class ->  Reducer.class

will not work.

Why is that?

Upvotes: 0

Views: 45

Answers (1)

TC1
TC1

Reputation: 1

I don't see why it wouldn't as long as the outputs are of the same class as the inputs. The default in the new API just writes out whatever you pass into it, it's implemented as

@SuppressWarnings("unchecked")
protected void reduce(KEYIN key, Iterable<VALUEIN> values, Context context
                       ) throws IOException, InterruptedException {
    for(VALUEIN value: values) {
      context.write((KEYOUT) key, (VALUEOUT) value);
    }
}

For the old API, it's an interface, and you can't directly instantiate an interface. If you're using that, then that's the reason it fails. Then again, the Mapper is an interface as well, and you shouldn't be able to instantiate it...

Upvotes: 1

Related Questions