CODEWITHSUNDEEP
javahadoopmapreduce
RRP
RRP

Reputation: 2853

Hadoop WordCount error

I am following the documentation found at this link

https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/MapReduceTutorial.html#Usage

When i try to compile for WordCount.java and create a jar, I get the following error

bin/hadoop com.sun.tools.javac.Main WordCount.java
Error: Could not find or load main class com.sun.tools.javac.Main

I have verified my $JAVA_HOME and $HADOOP_CLASSPATH in the hadoop-env.sh file and also verified to see if I have the jdk

Here are the contents from hadoop-env.sh

export JAVA_HOME="/Library/Java/JavaVirtualMachines/jdk1.8.0_111.jdk/Contents/Home/"
.......
.........
for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; do
  if [ "$HADOOP_CLASSPATH" ]; then
    export HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar"
  else
    export HADOOP_CLASSPATH=$f
  fi

I am not sure the reason behind error or if I am missing another key configuration?

Upvotes: 1

Views: 421

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191728

This doesn't make sense in that loop... Nor does checking the existence of the variable first

if [ "$HADOOP_CLASSPATH" ]; then
  export HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar"
else

You need to set HADOOP_CLASSPATH="$JAVA_HOME/lib/tools.jar", as the documentation says for that class to be found. And that class is only available in the JDK

But, you could just run javac command to compile code. Not sure why the docs have you calling that class.

How to compile a Hadoop program

$ javac -classpath ${HADOOP_CLASSPATH} -d WordCount/ WordCount.java

To create jar:

$ jar -cvf WordCount.jar -C WordCount/ .

To run:

$ hadoop jar WordCount.jar WordCount input/ output

Suggestion Please use Maven/Gradle to create proper JAR files, and an IDE to write code.

P.S. Not many people actually write plain MapReduce

Upvotes: 2

Related Questions