CODEWITHSUNDEEP
javahadoopjarcompiler-errors
seso
seso

Reputation: 31

hadoop compilation - in the dfs file

I want to compile hadoop example with the following commands, but an error occur: enter image description here

$ mkdir wordcount_classes
$ javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d wordcount_classes WordCount.java
$ jar -cvf /usr/joe/wordcount.jar -C wordcount_classes/ .

Assuming that:

    /usr/joe/wordcount/input - input directory in HDFS
    /usr/joe/wordcount/output - output directory in HDFS

Sample text-files as input:

$ bin/hadoop dfs -ls /usr/joe/wordcount/input/
/usr/joe/wordcount/input/file01
/usr/joe/wordcount/input/file02

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file01
Hello World Bye World

$ bin/hadoop dfs -cat /usr/joe/wordcount/input/file02
Hello Hadoop Goodbye Hadoop

Upvotes: 0

Views: 148

Answers (1)

Ben Watson
Ben Watson

Reputation: 5541

/usr/joe is local, as you can see when you execute the ls command on the first line. The second command requires input and output in HDFS locations, but /usr/joe doesn't exist on the HDFS. You need to move your data onto the HDFS and then execute the command. For example:

#This creates a folder "wordcount/input" in your HDFS home directory
hdfs dfs -mkdir -p wordcount/input
hdfs dfs -put /usr/joe/wordcount/input/* wordcount/input

Upvotes: 1

Related Questions