CODEWITHSUNDEEP
hadoopmapreduce
user7431472
user7431472

Reputation: 11

hadoop ERROR ON WORDCOUNT PROGRAM

 public static void main(String[] args) throws Exception 
 {
    Configuration conf = new Configuration();
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/core-site.xml"));
    conf.addResource(new Path("/usr/local/hadoop/etc/hadoop/hdfs-site.xml"));
    Job job = new Job();
    //job.setJarByClass(WordCount.class);
    job.setJobName("WordCounter");

    job.setJarByClass(WordCount.class);
    job.setMapperClass(TokenizerMapper.class);
    job.setCombinerClass(IntSumReducer.class);
    job.setReducerClass(IntSumReducer.class);
    job.setOutputKeyClass(Text.class);
    job.setOutputValueClass(IntWritable.class);
    FileInputFormat.addInputPath(job, new Path("/home/pramukh/eclipse/eclipseproj/hadoop/input/input.txt"));
    FileOutputFormat.setOutputPath(job, new Path("/home/pramukh/eclipse/eclipseproj/hadoop/output.txt"));
//System.exit(0);
    System.exit(job.waitForCompletion(true) ? 0 : 1);
  }

This is my java hadoop wordcount example, which gives the following error:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/io/DatumReader at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:348) at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1074) at org.apache.hadoop.io.serializer.SerializationFactory.add(SerializationFactory.java:69) at org.apache.hadoop.io.serializer.SerializationFactory.(SerializationFactory.java:62) at org.apache.hadoop.mapreduce.split.JobSplitWriter.writeNewSplits(JobSplitWriter.java:117) at org.apache.hadoop.mapreduce.split.JobSplitWriter.createSplitFiles(JobSplitWriter.java:74) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1030) at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1041) at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:959) at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:742) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912) at org.apache.hadoop.mapreduce.Job.submit(Job.java:500) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530) at hadoop.WordCount.main(WordCount.java:66) Caused by: java.lang.ClassNotFoundException: org.apache.avro.io.DatumReader at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331) at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Thanks in advance.

Upvotes: 1

Views: 733

Answers (1)

HbnKing
HbnKing

Reputation: 1882

If the rest of your code in this program is corrected; I suggest you check the path The OutputPath folder can't be exist
I tried your code changes the file path ,the code can run normally

FileInputFormat.addInputPath(job, new Path("hdfs://localhost:9000/wordcount/test.txt"));
    FileOutputFormat.setOutputPath(job, new Path("hdfs://localhost:9000/wordcount/out"));

Upvotes: 0

Related Questions