CODEWITHSUNDEEP
hadoophdfs
Rushik
Rushik

Reputation: 1131

HDFS corrupts namenode in my singe node experimental local setup

For my Hadoop experiments I've setup single node HDFS on localhost on my macbook. Everytime after my experiments I shutdown hadoop services (./bin/stop-all.sh).

Almost every time when I start my setup again (start-all.sh), I find my HDFS corrupted, I always need to format my namenode and restart services to get access of HDFS again.

I realized this as I stop getting namenode admin access (http://localhost:50070/) after startup.

During startup my namenode logs says "2012-11-29 15:27:43,160 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed. org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /private/tmp/hadoop-rupadhyay/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible."

I am not sure what I am missing here.

Upvotes: 2

Views: 1464

Answers (2)

Vbp
Vbp

Reputation: 1962

I had the same issue and a bit of google solved it, hadoop.tmp.dir in the core-site.xml is defaulted to /tmp/hadoop-username which gets cleaned on every reboot. you can change this to some other directory. An example will be

<configuration>
  <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:54310</value>
  </property>

  <property>
     <name>hadoop.tmp.dir</name>
     <value>/home/hadoop/hdfs-tmp</value>
  </property>
</configuration>

This link is helpful too Namenode not getting started

Upvotes: 4

Chris Gerken
Chris Gerken

Reputation: 16392

Using a tmp directory means that you have to be sure to clear the directory and reformat your name node every time you restart. This is because the OS may delete files at whim from the tmp directory leaving your hdfs in an unstable state.

Upvotes: 4

Related Questions