error when setting the job.setInputFormatClass

Question

I am facing certain error while setting my job for a custom InputFormat

Below Is my Code

package com.nline_delimiter;

import java.io.IOException;

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;



public class NL_driver {

public static void main(String [] args) throws IOException, InterruptedException, ClassNotFoundException
{
    Configuration conf=new Configuration(true);

    Job job_run =new Job(conf);

    job_run.setJobName("nline input format each line seperate wth delimiter");

    job_run.setJarByClass(NL_driver.class);

    job_run.setMapperClass(NL_mapper.class);
    job_run.setReducerClass(NL_reducer.class);
    job_run.setInputFormatClass(NL_inputformatter.class);;


    job_run.setMapOutputKeyClass(Text.class);
    job_run.setMapOutputValueClass(IntWritable.class);
    job_run.setOutputKeyClass(Text.class);
    job_run.setOutputValueClass(IntWritable.class);


    FileInputFormat.setInputPaths(job_run,new Path("/home/hduser/input_formatter_usage.txt"));
    FileOutputFormat.setOutputPath(job_run, new Path("/home/hduser/input_formatter_usage"));

    job_run.waitForCompletion(true);
}
}

The Line

job_run.setInputFormatClass(NL_inputformatter.class)

shows error

The NL_inputformatter is a custom Inputformatter class that extends the FileInputFormat

Is there something I need to import for the setInputFormatClass, because the default error check in Eclipse asks me to change the setInputFormatClass to setOutFormatClass but doesn't asks for any Import.

The Sourcecode for NL_inputformatter is below.

package com.nline_delimiter;

import java.io.IOException;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.FileSplit;
import org.apache.hadoop.mapred.InputSplit;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.mapred.RecordReader;
import org.apache.hadoop.mapred.Reporter;

public class NL_inputformatter extends FileInputFormat<Text, IntWritable>{

@Override
public RecordReader<Text, IntWritable> getRecordReader(InputSplit input,
        JobConf job_run, Reporter reporter) throws IOException {
    // TODO Auto-generated method stub
    System.out.println("I am Inside the NL_inputformatter class");
    reporter.setStatus(input.toString());
    return new NL_record_reader(job_run, (FileSplit)input);


}

}

Your help would be appreciated.

Answer 1

It's because you are using the FileInputFormat from the old Hadoop API with the new one. You have to change your import and your implementation:

import org.apache.hadoop.mapred.FileInputFormat;

to

import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;