CODEWITHSUNDEEP
hadoopmapreducehdfs
Kumar
Kumar

Reputation: 949

Trying to read data from Hadoop URL but unable to do so.I am getting error

The Contents of the ReadFromHadoopURL file are as follows:

package com.felix.hadoop.training;

import java.io.InputStream;
import java.net.URL;

import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;
import org.apache.hadoop.io.IOUtils;

//  URLCat
public class ReadFromHadoopURL {

  static {
        URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());
  }

  public static void main(String[] args) throws Exception {
    InputStream in = null;
    try {
      in = new URL(args[0]).openStream();
      IOUtils.copyBytes(in, System.out, 4096, false);
    } finally {
      IOUtils.closeStream(in);
    }
      }
    }

Then I created a jar file ReadFromHadoopURL.jar .

Then I ran this command

hadoop jar /home/training/Pradosh/ReadFromHadoopURL.jar ReadFromHadoopURL hdfs://localhost:54310/my/empfile

I am getting this error 

Exception in thread "main" java.lang.ClassNotFoundException: ReadFromHadoopURL
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:149)

Could you please help me to debug this?

Upvotes: 0

Views: 457

Answers (2)

Mike Park
Mike Park

Reputation: 10941

You probably just need to fully qualify your class name with the package name. ReadFromHadoopURL should be com.felix.hadoop.training.ReadFromHadoopURL.

hadoop jar /home/training/Pradosh/ReadFromHadoopURL.jar com.felix.hadoop.training.ReadFromHadoopURL hdfs://localhost:54310/my/empfile

Upvotes: 1

RickH
RickH

Reputation: 2486

To run with the hadoop jar command, you have to implement a class containing Map extends Mapper<> class (and optionally Reduce extends Reducer<>) as illustrated in the canonical WordCount Example.

The above looks like you're trying to implement a simple Java console app to cat a file stored in HDFS. If that's the case then you should run it with something like:

java -cp ./ReadFromHadoopURL.jar ReadFromHadoopURL hdfs://localhost:54310/my/empfile

Note: getting all of the Hadoop and Apache dependencies to work from the command line will likely require a much expanded cp/classpath.

Upvotes: 0

Related Questions