CODEWITHSUNDEEP
hadoop
Ansd
Ansd

Reputation: 1865

Hadoop job not working

I'm following this instructions to run hadoop:

http://wiki.apache.org/hadoop/Running_Hadoop_On_OS_X_10.5_64-bit_(Single-Node_Cluster)

however, I couldn't get this command to work:

hadoop-*/bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'

all what I get is:

Exception in thread "main" java.io.IOException: Error opening job jar: /Users/hadoop/hadoop-1.0.1/hadoop-examples-1.0.1.jargrep
    at org.apache.hadoop.util.RunJar.main(RunJar.java:90)
Caused by: java.util.zip.ZipException: error in opening zip file
    at java.util.zip.ZipFile.open(Native Method)
    at java.util.zip.ZipFile.<init>(ZipFile.java:127)
    at java.util.jar.JarFile.<init>(JarFile.java:135)
    at java.util.jar.JarFile.<init>(JarFile.java:72)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:88)

I added this to my hadoop-env.sh : 

export HADOOP_OPTS="-Djava.security.krb5.realm=OX.AC.UK -Djava.security.krb5.kdc=kdc0.ox.ac.uk:kdc1.ox.ac.uk"

but still the same error. Any clue guys?

Upvotes: 1

Views: 3698

Answers (2)

Simon O&#39;Hanlon
Simon O&#39;Hanlon

Reputation: 59970

I know this is old, but in case anyone else has the same problem and sees this SO question, I want to put up what I did to solve this, as it's very simple.

It looks like it's a typo in the example's instructions. If you look in the Hadoop distribution directory you will notice that the example file being referred to is called hadoop-examples-1.0.4.jar, or whatever version you are using.

So instead of:

hadoop-*/bin/hadoop jar hadoop-*-examples.jar grep input output 'dfs[a-z.]+'

try:

bin/hadoop jar hadoop-examples-*.jar grep input output 'dfs[a-z.]+'

Upvotes: 2

AvkashChauhan
AvkashChauhan

Reputation: 20556

When you run the following command:

hadoop-/bin/hadoop jar hadoop--examples.jar grep input output 'dfs[a-z.]+'

  • grep is the hadoop program which is part of example
  • input is the folder where your source data is and hope you have created it at HDFS
  • output is the folder which will be created as result.
  • 'dfs[a=-z.]+' is the regular options used with grep program

because the output is "Grep......." it seems to me that the actual sample application class is not available or missing some info when Hadoop command is running.. you would need to check that first and also look for regular expression if that applies with your input data.

Upvotes: 3

Related Questions