CODEWITHSUNDEEP
hadoophdfscloudera
cybertextron
cybertextron

Reputation: 10981

Invalid URI for NameNode address

I'm trying to set up a Cloudera Hadoop cluster, with a master node containing the namenode, secondarynamenode and jobtracker, and two others nodes containing the datanode and tasktracker. The Cloudera version is 4.6, the OS is ubuntu precise x64. Also, this cluster is being created from a AWS instance. ssh passwordless has been set as well, Java instalation Oracle-7.

Whenever I execute sudo service hadoop-hdfs-namenode start I get:

2014-05-14 05:08:38,023 FATAL org.apache.hadoop.hdfs.server.namenode.NameNode: Exception in namenode join
java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:329)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:317)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.getRpcServerAddress(NameNode.java:370)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.loginAsNameNodeUser(NameNode.java:422)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:442)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:621)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:606)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1177)
        at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1241)

My core-site.xml:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
   <property>
      <name>fs.defaultFS</name>
      <value>hdfs://<master-ip>:8020</value>
   </property>
</configuration>

mapred-site.xml:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
   <property>
      <name>mapred.job.tracker</name>
      <value>hdfs://<master-ip>:8021</value>
   </property>
</configuration>

hdfs-site.xml:

<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<!-- Put site-specific property overrides in this file. -->
<configuration>
   <property>
      <name>dfs.replication</name>
      <value>2</value>
   </property>
   <property>
      <name>dfs.permissions</name>
      <value>false</value>
   </property>
</configuration>

I tried using public ip, private-ip, public dns and fqdn, but the result is the same. The directory /etc/hadoop/conf.empty looks like:

-rw-r--r-- 1 root root   2998 Feb 26 10:21 capacity-scheduler.xml
-rw-r--r-- 1 root hadoop 1335 Feb 26 10:21 configuration.xsl
-rw-r--r-- 1 root root    233 Feb 26 10:21 container-executor.cfg
-rwxr-xr-x 1 root root    287 May 14 05:09 core-site.xml
-rwxr-xr-x 1 root root   2445 May 14 05:09 hadoop-env.sh
-rw-r--r-- 1 root hadoop 1774 Feb 26 10:21 hadoop-metrics2.properties
-rw-r--r-- 1 root hadoop 2490 Feb 26 10:21 hadoop-metrics.properties
-rw-r--r-- 1 root hadoop 9196 Feb 26 10:21 hadoop-policy.xml
-rwxr-xr-x 1 root root    332 May 14 05:09 hdfs-site.xml
-rw-r--r-- 1 root hadoop 8735 Feb 26 10:21 log4j.properties
-rw-r--r-- 1 root root   4113 Feb 26 10:21 mapred-queues.xml.template
-rwxr-xr-x 1 root root    290 May 14 05:09 mapred-site.xml
-rw-r--r-- 1 root root    178 Feb 26 10:21 mapred-site.xml.template
-rwxr-xr-x 1 root root     12 May 14 05:09 masters
-rwxr-xr-x 1 root root     29 May 14 05:09 slaves
-rw-r--r-- 1 root hadoop 2316 Feb 26 10:21 ssl-client.xml.example
-rw-r--r-- 1 root hadoop 2251 Feb 26 10:21 ssl-server.xml.example
-rw-r--r-- 1 root root   2513 Feb 26 10:21 yarn-env.sh
-rw-r--r-- 1 root root   2262 Feb 26 10:21 yarn-site.xml

and slaves lists the ip addresses of the two slave machines:

<slave1-ip>
<slave2-ip>

Executing 

update-alternatives --get-selections | grep hadoop
hadoop-conf                    auto     /etc/hadoop/conf.empty

I've done a lot of search, but didn't get anything that could help me fix my problem. Could someone offer any clue what's going on?

Upvotes: 11

Views: 22298

Answers (4)

Sonu
Sonu

Reputation: 732

I was facing the same issue and fixed by formatting the namenode. Below is the command:

hdfs namenode -format

core-site.xml entry is :

<configuration>
   <property>
      <name>fs.defaultFS</name>
      <value>hdfs://localhost:9000</value>
   </property>
</configuration>

That will definitely solve the problem.

Upvotes: 16

Nik Bates-Haus
Nik Bates-Haus

Reputation: 211

I ran into this same thing. I found I had to add a fs.defaultFS property to hdfs-site.xml to match the fs.defaultFS property in core-site.xml:

<property>
      <name>fs.defaultFS</name>
      <value>hdfs://<master-ip>:8020</value>
</property>

Once I added this, the secondary namenode started OK.

Upvotes: 4

pavanpenjandra
pavanpenjandra

Reputation: 1

Might be you had given a wrong syntax for dfs.datanode.data.dir or dfs.namenode.data.dir in hdfs-site.xml. If you miss / in the value you will get this error. Check the syntax of file:///home/hadoop/hdfs/

Upvotes: 0

Vishwanath S
Vishwanath S

Reputation: 37

Make sure you have set the HADOOP_PREFIX variable correctly as indicated in the link: http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html

Even i faced the same issue as yours and it got rectified by setting this variable

Upvotes: 1

Related Questions