CODEWITHSUNDEEP
hadoophdfshadoop2
user6119874
user6119874

Reputation: 105

what are the step by step process of starting hadoop

I'm new to hadoop. So can you please describe what exactly I'm doing here. P.S I received this steps from a friend of mine.

(1) hduser@soham-Inspiron-3521:/usr/local/hadoop/etc/hadoop$ /usr/local/hadoop/bin/hadoop namenode -format
Que 1) why do we need to format namenode each time and not datanode or others
Que 2) why are we using two diiferent path each time

(2) `hduser@soham-Inspiron-3521:/usr/local/hadoop/etc/hadoop$ /usr/local/hadoop/sbin/start-all.sh
Que 1) Do all processes need to be started from "sbin" folder

(3) jps Displays :

hduser@soham-Inspiron-3521:/usr/local/hadoop/etc/hadoop$ jps
7344 ResourceManager
15019 Jps
7187 SecondaryNameNode
6851 NameNode
7659 NodeManager

Que 1) What about taskTracker and jobTracker ?

Even localhost isnot displaying any DataNode (http://localhost:50070/dfshealth.html#tab-startup-progress)

P.S I know these are naive problems but I could not find any solution whatsoever that could solve this problem. Fast reply would be greatly appreciated. Thanks in advance.

Upvotes: 3

Views: 329

Answers (1)

PradeepKumbhar
PradeepKumbhar

Reputation: 3421

This is what I could say from the information you have provided:

(1) You don't have to format namenode each time you start the hadoop. It's a one time activity. Once you do it, then whenever you start hadoop next time, you just need to start HDFS (start-dfs.sh) and YARN (start-yarn.sh) services. [P.S. Don't use start-all.sh as it is deprecated]

About the second part of your question, "why are we using two different path each time", which 2 paths you are referring to?

(2) Yes, all processes need to be started from "sbin" folder of your hadoop installation (e.g. /usr/local/hadoop/sbin/).

(3) From jps output, it's clear that you are using hadoop 2.0 in which JobTracker and TaskTracker have corresponding equivalents (but not exactly) as ResourceManager and NodeManager respectively.

Your DataNode is not running. Check the log messages while starting hadoop services to know more about what's going wrong.

Upvotes: 2

Related Questions