CODEWITHSUNDEEP
javahadoopmapreduce
frazman
frazman

Reputation: 33213

newbie hadoop .. distributed cache

I have a mapper

public class BuildGraph{
   public void config(JobConf job){ <==this block doesnt seems to be exexcuting at all :(
    super.configure(job);
    this.currentId = job.getInt("currentId",0);
    if (this.currentId!=0){ 
    // I call a method from differnt class to build a distributed cache
    }
   }
  public void map(....){
....  
}

}

now the main code where this is called..

public void run( String params,curId){
 JobConf conf = new JobConf(classname.class);
 conf.setInt("currentId",299); <--note this i am setting the value here
 conf.setMapperClass(BuildGraph.class);
 //.... 
  JobClient.runJob(conf);
}

But the problem is config method in the code is not executing as though "currentId" returns 299 in main loop but it is not set at all in the mapper class. what am i doing wrong

Link to full code http://pastebin.com/DRPXUm62

Upvotes: 0

Views: 123

Answers (1)

Charles Menguy
Charles Menguy

Reputation: 41428

It looks like you are not using the right contract since you are not extending MapReduceBase and not implementing Mapper. Also the method should be called configure and not config. Try something like this:

public class BuildGraph extends MapReduceBase implements Mapper<K, V, K, V> {
    public void configure(JobConf job) {
        // this will get called once at the beginning of the task
    }

    public void map(...) {
        ...
    }
}

Upvotes: 1

Related Questions