CODEWITHSUNDEEP
hadoop
aqjune
aqjune

Reputation: 552

In Hadoop, How can I find which slave node is executing an attempt N?

I'm using Hadoop 1.2.1, and my hadoop application fails in doing Reduce. From Hadoop run I see messages like following : 

15/05/22 18:14:15 INFO mapred.JobClient:  map 0% reduce 0% 15/05/22
18:14:25 INFO mapred.JobClient:  map 100% reduce 0% 15/05/22 18:24:25
INFO mapred.JobClient:  map 0% reduce 0% 15/05/22 18:24:26 INFO
mapred.JobClient: Task Id : attempt_201505221804_0013_m_000000_0,
Status : FAILED Task attempt_201505221804_0013_m_000000_0 failed to
report status for 600 seconds. Killing! 15/05/22 18:24:35 INFO
mapred.JobClient:  map 100% reduce 0%

I'd like to see the log of attempt_201505221804_0013_m_000000_0, but it is too time-consuming to find which slave had executed attempt_201505221804_0013_m_000000_0.

Someone told me to use Hadoop web pages to find it, but there is some firewall on this cluster and I can't change the option because the cluster is fundamentally not owned by our group.

Is there any way to find in where this attempt was executed?

Upvotes: 0

Views: 50

Answers (1)

lukas
lukas

Reputation: 58

You should be able to find this information in the jobtracker logs which are by default under HADOOP_HOME/logs. This will contain entries looking similar to this:

INFO org.apache.hadoop.mapred.JobTracker: Adding task (MAP) 'attempt_201503262103_0001_m_000000_0' to tip task_201503262103_0001_m_000000, for tracker 'host'

You can search the file for the specific attempt id.

Upvotes: 1

Related Questions