CODEWITHSUNDEEP
javahadoophbasereducers
Bayu
Bayu

Reputation: 33

Hadoop application cannot find Reducer

I am trying to make a mapreduce application that reads from a Hbase table and writes the results of the job in a textfile. My driver code looks like this:

    Configuration conf = HBaseConfiguration.create();
    Job job = Job.getInstance (conf, "mr test");
    job.setJarByClass(Driverclass.class);
    job.setCombinerClass(reducername.class);
    job.setReducerClass(reducername.class);

    Scan scan = new Scan();
    scan.setCaching(500);            
    scan.setCacheBlocks(false);        

    String qualifier = "qualifname"; // comma seperated
    String family= "familyname";
    scan.addColumn(Bytes.toBytes(family), Bytes.toBytes(qualifier));

    TableMapReduceUtil.initTableMapperJob("tablename", 
                                           scan, 
                                           mappername.class, 
                                           Text.class, Text.class, 
                                           job);

when initTableMapperJob is invoked, I get a ClassNotFoundException: class reducername not found.

The class is defined in another java file, within the same package. I used almost the same configuration to try the usual wordcount example and worked fine. Then I changed the type of mapper and the way it is configured, and I get this error. Can someone help me?

Edit: the code of the reducer class is:

package mr.roadlevelmr;
import java.io.IOException;
import java.util.ArrayList;
import java.util.StringTokenizer;
import org.apache.hadoop.io.*;
import org.apache.hadoop.mapreduce.Reducer;
public class reducername extends Reducer <Text, Text, Text, Text>{
    private Text result= new Text();

    public void reduce (Text key, Iterable<Text> values, Context context) throws IOException, InterruptedException{
       ArrayList<String> means = new ArrayList<String>();
        for (Text val : values){
            means.add(String.valueOf(val.getBytes()));
        }
        result.set(newMean(means));
        context.write(key, result);
    }

Upvotes: 3

Views: 121

Answers (1)

Peter P
Peter P

Reputation: 501

you should use the Map reduce util as follows:

TableMapReduceUtil.initTableMapperJob("tablename", 
                                       scan, 
                                       mappername.class, 
                                       Text.class, Text.class, 
                                       job);Ok think I found the issue!

than add the reducer and combiner

job.setCombinerClass(reducername.class);
job.setReducerClass(reducername.class);
boolean b = job.waitForCompletion(true);

Instead of adding the reducer to the table mapper job

Upvotes: 2

Related Questions