CODEWITHSUNDEEP
javahadoopmapreduce
hailah
hailah

Reputation: 13

Hadoop class not found exception

I'm working on simple program on hadoop, I followed this tutorial steps: http://www.bogotobogo.com/Hadoop/BigData_hadoop_Creating_Java_Wordcount_Project_with_Eclipse_MapReduce2.php

even though I tried it on two different machines, it keeps showing this exception: 

Exception in thread "main" java.lang.ClassNotFoundException: test.java
at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:270)
at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

package pa2;
import org.apache.hadoop.conf.Configured;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.FileOutputFormat;
import org.apache.hadoop.mapred.FileInputFormat;
import org.apache.hadoop.mapred.JobClient;
import org.apache.hadoop.mapred.JobConf;
import org.apache.hadoop.util.Tool;
import org.apache.hadoop.util.ToolRunner;


public class test extends Configured implements Tool{


public int run(String[] args) throws Exception
{ if (args.length<2)
{
    System.out.println("plz give proper arguments");
    return -1;
}
      //creating a JobConf object and assigning a job name for identification purposes
      JobConf conf = new JobConf(test.class);

      FileInputFormat.setInputPaths(conf, new Path(args[0]));
      FileOutputFormat.setOutputPath(conf, new Path(args[1]));

      conf.setMapperClass(mapper.class);

      conf.setMapOutputKeyClass(Text.class);
      conf.setMapOutputValueClass(IntWritable.class);

      conf.setOutputKeyClass(Text.class);
      conf.setOutputValueClass(IntWritable.class);

      JobClient.runJob(conf);

      return 0;
}


public static void main(String[] args) throws Exception
{
      // this main function will call run method defined above.
  int exitcode = ToolRunner.run(new test(),args);
      System.exit(exitcode);
}
}

can you please tell me what is wrong here?

update:

mapper class:

package pa2;
import java.io.IOException;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapred.MapReduceBase;
import org.apache.hadoop.mapred.Mapper;
import org.apache.hadoop.mapred.OutputCollector;
import org.apache.hadoop.mapred.Reporter;


public class mapper extends MapReduceBase 
        implements Mapper<LongWritable,Text, Text, IntWritable>
{
            public void map(LongWritable Key, Text value,
            OutputCollector<Text, IntWritable> output, Reporter r)
            throws IOException {


            int i=0;
            String [] array = new String [50];


                        String name;
                        String year;
                        String s=value.toString();

                        for (String word:s.split(",")){

                   word = s.substring(0, s.indexOf(",")+1);
                   year= word.substring(0, s.indexOf(",")+1);
                   name=word.substring(s.indexOf(",")+1);
                   int theyear= Integer.parseInt(year);


                   if(theyear<2000){
                        array[i] =name;
                        output.collect(new Text(word),  new IntWritable(1));

                        i++;}

                    }       
    }
}

I haven't written the reducer class. I exported the project as jar file,and I made a text file called movies to be the input of the program. then wrote this in the terminal:

[cloudera@quickstart ~]$ cd workspace
[cloudera@quickstart workspace]$ ls
pa2  pa2.jar  training
[cloudera@quickstart workspace]$ hadoop jar pa2.jar test movies.txt output.txt
Exception in thread "main" java.lang.ClassNotFoundException: test
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:270)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:214)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Upvotes: 1

Views: 1828

Answers (2)

OneCricketeer
OneCricketeer

Reputation: 191743

No guarantees this is the solution to the immediate problem, but 

package pa2;

This is appended to the class name. In other words, the fully-qualified class name is pa2.test.

So, try

hadoop jar ~/workspace/pa2.jar pa2.test input output

If you used the default package like that tutorial showed, you wouldn't need to specify the package on the command line.

Upvotes: 1

kashmoney
kashmoney

Reputation: 97

The actual name of your map class should be provided here 

conf.setMapperClass(mapper.class);

If you are trying to use the default map class, then write "Mapper.class".

Upvotes: 0

Related Questions