CODEWITHSUNDEEP
hadoop
Ananda
Ananda

Reputation: 1572

Hadoop 2.0.2 multinode cluster running issue?

I am running a multi node hadoop 2.0.2 cluster with MRv2. My problem is at a time this particular page and fsck command only shows 3 live nodes:

enter image description here

Although this page shows actual no. of live nodes

enter image description here

Another problem is some times some datanodes are getting shutdown automatically. How can I know the pblm and why same cluster is showing two different live nodes?

Upvotes: 2

Views: 1202

Answers (2)

dilshad
dilshad

Reputation: 754

Add Some configuration should be done in yarn-site.xml to let the nodemanager know where is the resource manager. Specifically, I added this property into yarn-site.xml:

<property> <name>yarn.resourcemanager.hostname</name> <value>master</value> </property>

Reason: the default value in yarn-default.xml is 0.0.0.0, and many properties use this hostname to contact resource manager, such as

<property> <name>yarn.resourcemanager.address</name> <value>${yarn.resourcemanager.hostname}:8032</value> </property>

Upvotes: 0

user568109
user568109

Reputation: 47993

About the web interfaces you are using http://localhost:50070/dfshealth.jsp is for NameNode WebUI and http://localhost:8088/cluster is for ResourceManager WebUI.

Although this page shows actual no. of live nodes

Actually it does not show live nodes. If you see the Last Health Updates for each node, you will find that three of the nodes slave-1, 2 and 3 have been updated last on 16th March 2013 and the rest have been updated last on 15th March 2013. This tallies with what you see on Namenode Interface at 50070. Only 3 are active now.

Another problem is some times some datanodes are getting shutdown automatically.

Four of your nodes went down around 01:00:00 on 15th March 2013 (last update). So I would investigate your datanode logfiles for that time as why they shut down. You can find them either inside $HADOOP_HOME/logs/ or /var/log/hadoop*, if you have not set HADOOP_LOG_DIR environment variable

Upvotes: 1

Related Questions