CODEWITHSUNDEEP
hadoophdfs
acushner
acushner

Reputation: 9946

determine which default namenode and namenode_port "hadoop fs -ls" is using

I'm running the below command:

hadoop fs -ls

I know that this command really corresponds to something like the below:

hadoop fs -ls hdfs://<namenode>:<namenode_port>

How do I determine these values of namenode and namenode_port?

I've already tried examining environment variables and looking at the documentation, but I couldn't find anything to do exactly this.

Upvotes: 1

Views: 1029

Answers (1)

SachinJose
SachinJose

Reputation: 8522

You need to refer hadoop configuration file core-site.xml for <namenode> and <namenode_port> values. Search for configuration property fs.defaultFS(latest) or fs.default.name(depricated).

core-site.xml file can be located in /etc/hadoop/conf or $HADOOP_HOME/etc/hadoop/conf locations.

Edit

There is a dynamic way to do this. Below hadoop command will give HDFS URI.

hdfs getconf -confKey "fs.defaultFS"

However, every time hadoop commands get executed, hadoop loads client configurations xml files. If you wanted to test this out, create a separate client configurations in some other directories by copying contents of /etc/hadoop/conf to new directory( let's say /home//hadoop-conf, it should contain core-site.xml, hdfs-site.xml, mapred-site.xml), override the environment variable using the command export HADOOP_CONF_DIR=/home/<USER>/hadoop-conf, update core-site.xml files, then test using hadoop command. Remember this is only for testing purpose, unset the environment variable ( unset HADOOP_CONF_DIR) once you are done with your testing.

Upvotes: 2

Related Questions