CODEWITHSUNDEEP
javaspringmaventomcathadoop
Jit B
Jit B

Reputation: 1246

Hadoop Configured getConf() returning null

I have a Spring MVC application running on tomcat which submits MapReduce jobs and analyzes results. My Spring Batch tasklet is able to successfully call an MR driver class and run the job. The driver class extends Configured and implements Tool and is easily able to manipulate HDFS files. The maven module containing the driver class and MR code is added as a dependency to the webapp module.

For analysis, I created a new class in the webapp module which extends Configured. This class is supposed to read an HDFS file and analyze it. However when I try to create the FileSystem object I am getting a null pointer exception.

public class ReportAnalyzer extends Configured{

    public void analyze(String path) throws Exception{
        FileSystem hdfs=FileSystem.get(getConf()); <-- NPE

        //create Path, etc.
    }
}

Is there anything else that needs to be done in order to get the FileSystem object? the hadoop dependencies are added to the webapp via the mapreduce module.

Upvotes: 2

Views: 1927

Answers (1)

user3122114
user3122114

Reputation: 131

You either have to implement Configured(Configuration conf) constructor

public ReportAnalyzer(Configuration conf){
    super(conf);
}

or use setConf() before calling analyze().

Upvotes: 1

Related Questions