Reputation: 563
Hi everyone today i setup a hadoop Multi-Node Cluster following this Tutorials http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-multi-node-cluster/ when i got ready to run the example program it gave me this
11/07/31 13:44:30 INFO input.FileInputFormat: Total input paths to process : 4
11/07/31 13:44:31 INFO mapred.JobClient: Running job: job_201107311331_0001
11/07/31 13:44:32 INFO mapred.JobClient: map 0% reduce 0%
11/07/31 13:44:55 INFO mapred.JobClient: Task Id : attempt_201107311331_0001_m_000002_0, Status : FAILED
java.io.FileNotFoundException: File does not exist: /user/hduser/gutenberg/gutenberg
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1590)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1581)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:423)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:762)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
11/07/31 13:44:56 INFO mapred.JobClient: map 25% reduce 0%
11/07/31 13:44:58 INFO mapred.JobClient: map 50% reduce 0%
11/07/31 13:45:10 INFO mapred.JobClient: map 75% reduce 8%
11/07/31 13:45:22 INFO mapred.JobClient: Task Id : attempt_201107311331_0001_m_000002_1, Status : FAILED
java.io.FileNotFoundException: File does not exist: /user/hduser/gutenberg/gutenberg
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1590)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1581)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:423)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:762)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
11/07/31 13:45:22 WARN mapred.JobClient: Error reading task outputACRS-HCN1
11/07/31 13:45:22 WARN mapred.JobClient: Error reading task outputACRS-HCN1
11/07/31 13:45:29 INFO mapred.JobClient: Task Id : attempt_201107311331_0001_m_000002_2, Status : FAILED
java.io.FileNotFoundException: File does not exist: /user/hduser/gutenberg/gutenberg
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.openInfo(DFSClient.java:1590)
at org.apache.hadoop.hdfs.DFSClient$DFSInputStream.<init>(DFSClient.java:1581)
at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:423)
at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:187)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:400)
at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:67)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:521)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:762)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:369)
at org.apache.hadoop.mapred.Child$4.run(Child.java:259)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
at org.apache.hadoop.mapred.Child.main(Child.java:253)
11/07/31 13:45:29 WARN mapred.JobClient: Error reading task outputACRS-HCN1
11/07/31 13:45:29 WARN mapred.JobClient: Error reading task outputACRS-HCN1
11/07/31 13:45:44 INFO mapred.JobClient: Job complete: job_201107311331_0001
11/07/31 13:45:44 INFO mapred.JobClient: Counters: 19
11/07/31 13:45:44 INFO mapred.JobClient: Job Counters
11/07/31 13:45:44 INFO mapred.JobClient: Launched reduce tasks=1
11/07/31 13:45:44 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=94857
11/07/31 13:45:44 INFO mapred.JobClient: Total time spent by all reduces waiting after reserving slots (ms)=0
11/07/31 13:45:44 INFO mapred.JobClient: Total time spent by all maps waiting after reserving slots (ms)=0
11/07/31 13:45:44 INFO mapred.JobClient: Launched map tasks=7
11/07/31 13:45:44 INFO mapred.JobClient: Data-local map tasks=1
11/07/31 13:45:44 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=45085
11/07/31 13:45:44 INFO mapred.JobClient: Failed map tasks=1
11/07/31 13:45:44 INFO mapred.JobClient: FileSystemCounters
11/07/31 13:45:44 INFO mapred.JobClient: HDFS_BYTES_READ=662018
11/07/31 13:45:44 INFO mapred.JobClient: FILE_BYTES_WRITTEN=330575
11/07/31 13:45:44 INFO mapred.JobClient: File Input Format Counters
11/07/31 13:45:44 INFO mapred.JobClient: Bytes Read=661664
11/07/31 13:45:44 INFO mapred.JobClient: Map-Reduce Framework
11/07/31 13:45:44 INFO mapred.JobClient: Map output materialized bytes=267146
11/07/31 13:45:44 INFO mapred.JobClient: Combine output records=18043
11/07/31 13:45:44 INFO mapred.JobClient: Map input records=12761
11/07/31 13:45:44 INFO mapred.JobClient: Spilled Records=18043
11/07/31 13:45:44 INFO mapred.JobClient: Map output bytes=1086767
11/07/31 13:45:44 INFO mapred.JobClient: Combine input records=109848
11/07/31 13:45:44 INFO mapred.JobClient: Map output records=109848
11/07/31 13:45:44 INFO mapred.JobClient: SPLIT_RAW_BYTES=354
can some on tell me what is wrong
java.io.FileNotFoundException: File does not exist: /user/hduser/gutenberg/gutenberg
Upvotes: 0
Views: 2444
Reputation: 563
Ok everyone i found out what the problem was i had to add a *
to
bin/hadoop jar hadoop*examples*.jar wordcount /user/hduser/gutenberg /user/hduser/gutenberg-output
This is how it should have been in the Tutorial
bin/hadoop jar hadoop*examples*.jar wordcount /user/hduser/gutenberg/* /user/hduse/gutenberg-output3
Upvotes: 1