CODEWITHSUNDEEP
javahadoopmapreduce
Kumar Harsh
Kumar Harsh

Reputation: 453

Reducer code is code is not executing?

In my driver class I am running two jobs, my first job is working as expected but in my second job the reducer class is not executing.
Below is my driver class(JOb2 configurations):

if(job.waitForCompletion(true)){
            Configuration conf2 = new Configuration();
            Job job2 = Job.getInstance(conf2);

            MultipleInputs.addInputPath(job2, inOutPath, TextInputFormat.class, CombinedUserRatingMapper.class);
            MultipleInputs.addInputPath(job2, new Path(oArgs[3]), TextInputFormat.class, MovieMapper.class);
            FileOutputFormat.setOutputPath(job2, new Path(oArgs[4]));
            job2.setReducerClass(GenreCombinedReducer.class);       
            job2.setJarByClass(GroupedAnalysisDriver.class);
            job2.setNumReduceTasks(1);
            job2.setMapOutputKeyClass(IntWritable.class);
            job2.setMapOutputValueClass(Text.class);
            job2.setOutputKeyClass(Text.class);
            job2.setOutputValueClass(Text.class);

            System.out.println(job2.waitForCompletion(true)?0:1);
        }

Below are my both Mappers:
Mapper 1:

public class CombinedUserRatingMapper extends Mapper<LongWritable, Text, IntWritable, Text> {
//age   occ mid rating
private IntWritable mid = new IntWritable();
private Text val = new Text();

public void map(LongWritable key,Text value,Context context) throws IOException, InterruptedException{
    String[] token = value.toString().split("\t");
    int  i= Integer.parseInt(token[2]);
    mid.set(i);
    val.set("C"+"\t"+token[0]+"\t"+token[1]+"\t"+token[3]);
    context.write(mid, val);
}           //  mid     age occ     rating

}

Second Mapper:

public class MovieMapper extends Mapper<LongWritable, Text, IntWritable, Text> {


public void map(LongWritable key, Text value,Context context) throws IOException, InterruptedException {
    // movies.dat - 1::Toy Story (1995)::Animation|Children's|Comedy
    //              id  name                genre
    String[] token = value.toString().split("::");
        int id = Integer.parseInt(token[0]);  //getting ID and name
        String name = token[1];
        String genre = token[2];
        context.write(new IntWritable(id), new Text("G\t"+genre));
                        //mid               G   genre
}
}

And Finally the Reduce Class:

public class GenreCombinedReducer extends Reducer<IntWritable, Text, Text, Text>{
//  mid     G   genre
//  mid     C   age occ     rating
private Text outvalue=new Text();

public void Reduce(IntWritable key, Iterable<Text> values,Context context) throws IOException, InterruptedException{
    String genre = "";
    String combineInfo = "" ;
    for(Text val : values){
        if(val.charAt(0)=='G'){
            genre = val.toString().split("\t")[1];

        }else if(val.charAt(0)=='C'){
            String[] CI = val.toString().split("\t");
            combineInfo = CI[1]+CI[2]+CI[3];

        }
    }
    context.write(new Text(combineInfo),new Text(genre));
}


}

In output file I am getting the output of mapper class. I also tried debugging but cursor never came in Reducer class.

Upvotes: 0

Views: 52

Answers (1)

Liju John
Liju John

Reputation: 1876

In your Reducer class , you should override "reduce" method instead of "Reduce" . (Java is case sensitive )

Upvotes: 3

Related Questions