CODEWITHSUNDEEP
hadoophdfsnamenode
Faye
Faye

Reputation: 127

Namenode daemon not starting properly

I have just started learning hadoop from the book Hadoop: The definitive guide.

I followed the tutorial for Hadoop installation in Pseudodistribution mode. I enabled the passwordless login to ssh. Formatted the hdfs filesystem before using it for the first time. It started successfully for the first time. After that I copied a text file using copyFromLocal to HDFS and everything went fine. But if I restart the system and start the daemons again and look at the web UI , only YARN is started successfully.

When I issue the stop-dfs.sh commmand I get 

Stopping namenodes on [localhost]
localhost: no namenode to stop
localhost: stopping datanode
Stopping secondary namenodes [0.0.0.0]
0.0.0.0: stopping secondarynamenode

If I format the hdfs file system again and then try starting the daemons then they all start successfully.

Here are my configuration files.Exactly as what is told in hadoop definitive guide book.

hdfs-site.xml

<configuration>
    <property>
        <name>dfs.replication</name>    
        <value>1</value>
    </property>

</configuration>

core-site.xml

<configuration>
    <property>
        <name>fs.defaultFS</name>
        <value>hdfs://localhost/</value>
    </property>
</configuration>

mapred-site.xml

<configuration>
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

</configuration>

yarn-site.xml

<configuration>

<!-- Site specific YARN configuration properties -->
    <property>
        <name>yarn.resourcemanager.hostname</name>
        <value>localhost</value>    
    </property>

    <property>
        <name>yarn.nodemanager.aux-services</name>
        <value>mapreduce_shuffle</value>
    </property>
</configuration>

This is the error in the namenode log file

WARN org.apache.hadoop.hdfs.server.common.Storage: Storage directory /tmp/hadoop/dfs/name does not exist
 WARN org.apache.hadoop.hdfs.server.namenode.FSNamesystem: Encountered exception loading fsimage
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /tmp/hadoop/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs(FSImage.java:327)
    at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:215)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFSImage(FSNamesystem.java:975)
    at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.loadFromDisk(FSNamesystem.java:681)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.loadNamesystem(NameNode.java:585)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:645)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:812)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:796)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1493)
    at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1559)

This is from mapred log 

Caused by: java.net.ConnectException: Connection refused
    at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:744)
    at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:531)
    at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:495)
    at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:614)
    at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:712)
    at org.apache.hadoop.ipc.Client$Connection.access$2900(Client.java:375)
    at org.apache.hadoop.ipc.Client.getConnection(Client.java:1528)
    at org.apache.hadoop.ipc.Client.call(Client.java:1451)
    ... 33 more

I visited apache hadoop : connection refused which says

I found there is an entry in my /etc/hosts, but if I remove it my sudo breaks causing error sudo: unable to resolve host . What should I append in /etc/hosts if not remove my hostname mapped to 127.0.1.1

I cannot understand what is the root cause of this problem.

Upvotes: 0

Views: 1122

Answers (2)

Totoro
Totoro

Reputation: 1347

Well it says in your Namenode log file that default storage of your namenode directory is /tmp/hadoop. The /tmp directory is formatted in linux on reboot by some systems. So it must be the problem.

You need to change your default namenode and datanode directory by changing your hdfs-site.xml configuration file.

Add this in your hdfs-site.xml

<property>
        <name>dfs.namenode.name.dir</name>
        <value>file:///home/"your-user-name"/hadoop</value> 
    </property>
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:///home/"your-user-name"/datanode</value>   
    </property>

After this format your namenode by hdfs namenode -format command.

I think this will end your problem.

Upvotes: 2

Nishu Tayal
Nishu Tayal

Reputation: 20880

If configuration file is not a problem, please try following:

1.first delete all contents from temporary folder: 

 rm -Rf <tmp dir> (my was /usr/local/hadoop/tmp)

2.format the namenode: 

bin/hadoop namenode -format

3.start all processes again:

bin/start-all.sh

Upvotes: 0

Related Questions