CODEWITHSUNDEEP
javahadoopmapreduce
asembereng
asembereng

Reputation: 675

Sending a variable to the Mapper Class

I am trying to get an input from the user and pass it to my mapper class that I have created but whenever the value always initialises to zero instead of using the actual value the user input.

How can make sure that whenever I get the variable it always maintain the same value. I have noticed that job1.setMapperClass(Parallel_for.class); creates an instance of the class hence forcing the variable to reinitialize to its original value. Below is the link to the two classes. I am trying to get the value of times from RunnerTool class.

Link to Java TestFor class

Link to RunnerTool class

// setup method in the Mapper

@Override
public void setup(Context context) {
    int defaultValue = 1;
    times = context.getConfiguration().getInt("parallel_for_iteration", defaultValue );
    LOG.info(context.getConfiguration().get("parallel_for_iteration") + " Actually name from the commandline");
    LOG.info(times + " Actually number of iteration from the commandline");
}

// RunnerTools class

conf.setInt(ITERATION, times);

Upvotes: 0

Views: 4211

Answers (2)

David Gruzman
David Gruzman

Reputation: 8088

You should note that mapper class will be recreated on many cluster nodes so any initalization done to the instance of the mapper class when running the job will not affect other nodes. Technically relevant jar file/s will be distributed among nodes and then mappers will be created there. So as pointed in the answer above, the only way to pass information to the mappers is using Configuration class.

Upvotes: 1

Thomas Jungblut
Thomas Jungblut

Reputation: 20969

Mapper get's initialized by reflection, so you can not let the user interact with the mapper class.

Instead you have your Configuration object, which you have to provide if you're setting up your job. There you can set this using conf.set("YOUR KEY", "YOUR VALUE"). In your Mapper class you can override a method called setup(Context context), there you can get the value using context.getConfiguration().get("YOUR KEY"). And maybe save to your mapper local variable.

Upvotes: 0

Related Questions