CODEWITHSUNDEEP
javahadoopmapreduceoverridingreduce
xPeaWhyTee
xPeaWhyTee

Reputation: 85

Hadoop - @Override error for mappers and reducers

I've been working on a MapReduce program and I've hit a roadblock and need some help. I have a program that runs 3 jobs (job #2 gets ran 5 times within a for loop) and it seems like a few of my mappers and reducers are not properly defined. Upon compiling I keep getting "method does not override or implement a method from a supertype" errors.

Here's the basic structure of my program:

Job 1:

FirstMapper
No Reducer

Job 2:

Second Mapper
First Reducer

Job 3:

Final Mapper
Final Reducer

And here's how I have my mappers and reducers defined:

public static class FirstMapper extends Mapper<Object, Text, LongWritable, Vertex>  {
        @Override
        public void map(Object key, Text value, Context context) throws IOException, InterruptedException {}


public static class SecondMapper extends Mapper<LongWritable, Vertex, LongWritable, Vertex> {
        @Override
        public void map(long key, Vertex value, Context context) throws IOException, InterruptedException {}


public static class FirstReducer extends Reducer<LongWritable, Vertex, LongWritable, Vertex> {
        @Override
        public void reduce(Long key, Iterable<Vertex> values, Context context) throws IOException, InterruptedException {}


public static class FinalMapper extends Mapper<LongWritable, Vertex, LongWritable, LongWritable> {
        @Override
        public void map(Long key, Vertex value, Context context) throws IOException, InterruptedException {}


public static class FinalReducer extends Reducer<LongWritable, LongWritable, LongWritable, LongWritable> {
        @Override
        public void reduce(Long key, Iterable<LongWritable> values, Context context) throws IOException, InterruptedException {}

FirstMapper appears to be ok as it doesn't cause an error but the other 4 do and I can't figure out why because the method signatures look correct. I initially thought maybe it was because of my custom Vertex class but it implements Writable so I'm not sure why this would be causing an issue. Any and all help would be greatly appreciated. Thanks

Upvotes: 4

Views: 839

Answers (1)

Binary Nerd
Binary Nerd

Reputation: 13937

Your keys need to implement the WritableComparable interface and match the type you specify in the classes Mapper and Reducer signature. For example in the second one you have:

Mapper<LongWritable, Vertex, LongWritable, Vertex>

But then in the map method you use long. This needs to match the types you specify in the signature, ie LongWritable. So the parameters of the second map need to look like:

map(LongWritable key, Vertex value, Context context)

All of the last four classes have this problem.

Upvotes: 1

Related Questions