Hadoop SecondaryNameNode Issues

Question

Running Hadoop Version 1.2.1 using Ubuntu VM 4 VM's 1. hadoop-NN ( Name Node) 2. hadoop-snn ( Secondary Name Node) 3. hadoop-dn01 ( data node 1) 4. hadoop-dn02 ( data node 2) All process starts using start-all.sh

I don't see edit happenings in Secondary Name Node which means that fsiamge on secondary is not getting updated. LOg file on SecondaryNameNode shows following error.

2015-02-04 13:16:12,083 INFO org.apache.hadoop.hdfs.server.common.Storage: Number of files = 50 2015-02-04 13:16:12,086 INFO org.apache.hadoop.hdfs.server.common.Storage: Number of files under construction = 0 2015-02-04 13:16:12,087 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Start loading edits file /tmp/hadoop-hadoop/dfs/namesecondary/current/edits 2015-02-04 13:16:12,088 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: EOF of /tmp/hadoop-hadoop/dfs/namesecondary/current/edits, reached end of edit log Number of transactions found: 8. Bytes read: 740 2015-02-04 13:16:12,088 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Edits file /tmp/hadoop-hadoop/dfs/namesecondary/current/edits of size 740 edits # 8 loaded in 0 seconds. 2015-02-04 13:16:12,088 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: Number of transactions: 0 Total time for transactions(ms): 0 Number of transactions batched in Syncs: 0 Number of syncs: 0 SyncTimes(ms): 0 2015-02-04 13:16:12,128 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log: position=740, editlog=/tmp/hadoop-hadoop/dfs/namesecondary/current/edits 2015-02-04 13:16:12,128 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate to 740, editlog=/tmp/hadoop-hadoop/dfs/namesecondary/current/edits 2015-02-04 13:16:12,130 INFO org.apache.hadoop.hdfs.server.common.Storage: Image file /tmp/hadoop-hadoop/dfs/namesecondary/current/fsimage of size 5124 bytes saved in 0 seconds. 2015-02-04 13:16:12,229 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: closing edit log: position=4, editlog=/tmp/hadoop-hadoop/dfs/namesecondary/current/edits 2015-02-04 13:16:12,230 INFO org.apache.hadoop.hdfs.server.namenode.FSEditLog: close success: truncate to 4, editlog=/tmp/hadoop-hadoop/dfs/namesecondary/current/edits 2015-02-04 13:16:12,485 INFO org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL hadoop-nn:50070putimage=1&port=50090&machine=0.0.0.0&token=-41:307905665:0:1423080068000:1423079764851&newChecksum=9bbe4619db3323211ed473f3f8acb7a9 2015-02-04 13:16:12,485 INFO org.apache.hadoop.hdfs.server.namenode.TransferFsImage: Opening connection to http://hadoop-nn:50070/getimage?putimage=1&port=50090&machine=0.0.0.0&token=-41:307905665:0:1423080068000:1423079764851&newChecksum=9bbe4619db3323211ed473f3f8acb7a9 2015-02-04 13:16:12,489 ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Exception in doCheckpoint: 2015-02-04 13:16:12,490 ERROR org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: java.io.FileNotFoundException: http://hadoop-nn:50070/getimage?putimage=1&port=50090&machine=0.0.0.0&token=-41:307905665:0:1423080068000:1423079764851&newChecksum=9bbe4619db3323211ed473f3f8acb7a9 at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1624) at org.apache.hadoop.hdfs.server.namenode.TransferFsImage.getFileClient(TransferFsImage.java:177) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.putFSImage(SecondaryNameNode.java:462) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doCheckpoint(SecondaryNameNode.java:525) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.doWork(SecondaryNameNode.java:396) at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.run(SecondaryNameNode.java:360) at java.lang.Thread.run(Thread.java:745)

Answer 1

<property> <name>dfs.secondary.http.address</name> <value>hadoop-snn:50090</value> </property> Adding this tag in hdfs-site.xml solves issue.