CODEWITHSUNDEEP
hadoopmapreduce
Jon Cardoso-Silva
Jon Cardoso-Silva

Reputation: 990

Error when running Hadoop Map Reduce for map-only job

I want to run a map-only job in Hadoop MapReduce, here's my code:

    Configuration conf = new Configuration();
    Job job = new Job(conf);
    job.setJobName("import");

    job.setMapperClass(Map.class);//Custom Mapper
    job.setInputFormatClass(TextInputFormat.class);
    job.setNumReduceTasks(0);

    TextInputFormat.setInputPaths(job, new Path("/home/jonathan/input"));

But I get the error:

13/07/17 18:22:48 ERROR security.UserGroupInformation: PriviledgedActionException
as: jonathan cause:org.apache.hadoop.mapred.InvalidJobConfException: 
Output directory not set. 
Exception in thread "main" org.apache.hadoop.mapred.InvalidJobConfException: 
 Output directory not set.

Then I tried to use this:

job.setOutputFormatClass(org.apache.hadoop.mapred.lib.NullOutputFormat.class);

But it gives me a compilation error:

java: method setOutputFormatClass in class org.apache.hadoop.mapreduce.Job 
  cannot be applied to given types;
  required: java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat>
  found: java.lang.Class<org.apache.hadoop.mapred.lib.NullOutputFormat>
  reason: actual argument java.lang.Class
  <org.apache.hadoop.mapred.lib.NullOutputFormat> cannot be converted to 
  java.lang.Class<? extends org.apache.hadoop.mapreduce.OutputFormat> 
  by method invocation conversion

What am I doing wrong?

Upvotes: 1

Views: 4161

Answers (1)

Sean Owen
Sean Owen

Reputation: 66891

Map-only jobs still need an output location specified. As the error says, you're not specifying this.

I think you mean that your job produces no output at all. Hadoop still wants you to specify an output location, though nothing need be written.

You want org.apache.hadoop.mapreduce.lib.output.NullOutputFormat not org.apache.hadoop.mapred.lib.NullOutputFormat, which is what the second error indicates though it's subtle.

Upvotes: 6

Related Questions