CODEWITHSUNDEEP
javahadoop
Raunak Agarwal
Raunak Agarwal

Reputation: 7238

Hadoop: TaskTracker and JobTracker don't start with start-dfs.sh

I am trying to set up hadoop 0.22.0 on my laptop for learning purpose using this link

http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

When I run the script start-dfs.sh this is the ouput

starting namenode, logging to /usr/local/hadoop/bin/../logs/hadoop-raunak-namenode-ubuntu.out
localhost: starting datanode, logging to /usr/local/hadoop/bin/../logs/hadoop-raunak-datanode-ubuntu.out
localhost: starting secondarynamenode, logging to /usr/local/hadoop/bin/../logs/hadoop-raunak-secondarynamenode-ubuntu.out

The output of jps is:

18106 Jps
17269 NameNode
17556 DataNode
17845 SecondaryNameNode

But I can't see the TaskTracker or JobTracker running. So, can anyone please tell me what could I be missing. I am a total newbie and don't know what information is required. So, please let me know.

Upvotes: 0

Views: 13349

Answers (4)

Shubham Sharma
Shubham Sharma

Reputation: 2793

JobTracker and TaskTracker are 2 essential process involved in MapReduce execution in MRv1 (or Hadoop version 1). Both processes are now deprecated in MRv2 (or Hadoop version 2) and replaced by Resource Manager, Application Master and Node Manager Daemons. If you will just use start-all.sh then it will start your following services .

7120 ResourceManager
6787 DataNode
6981 SecondaryNameNode
7573 Jps
6678 NameNode
7229 NodeManager

Upvotes: 2

natcomp256
natcomp256

Reputation: 706

In case you are using the newer version of hadoop, you might find it a little different from the tutorial you mentioned. Now all the conf/ files will be in hadoop-installation/etc/hadoop directory.

All the hadoop daemons will be in sbin/ directory.

Also, when you run sbin/start-all.sh the script will say that this command is deprecated and will ask you to run start-dfs.sh and start-yarn.sh in my case.

start-dfs.sh started the namenode on master and data-nodes on slave and secondarynamenode on localhost.

start-yarn.sh started two new processes namely ResourceManager and NodeManager. Therefore I guess that the ResourceManager is the JobTracker and NodeManager is the TaskTracker in this case.

Upvotes: 4

Chris White
Chris White

Reputation: 30089

I agree with Chris Gerken's comment, you're running the start-dfs.sh script, which only starts the HDFS services. You'll need to run start-mapred.sh if you want the map reduce services (job tracker and task trackers).

Also, the deprecated message from start-all.sh hints towards this:

echo "This script is Deprecated. Instead use start-dfs.sh and start-mapred.sh"

Upvotes: 7

shazin
shazin

Reputation: 21923

Use the JobTracker web interface default is http://localhost:50030/ and TaskTracker web interface default is http://localhost:50060/. May be it is not showing in JPS for some reason.

Upvotes: 1

Related Questions