CODEWITHSUNDEEP
hadoophdfs
Vignesh Prajapati
Vignesh Prajapati

Reputation: 2350

How to check the distributed data over hdfs

we know, Hadoop replicates the data across several data nodes in hdfs, is there a command for checking the distributed data over different nodes.

Upvotes: 1

Views: 1902

Answers (3)

Mike Park
Mike Park

Reputation: 10941

I think you might be looking for this command

hdfs fsck /hdfs/path/to/data -files -blocks -locations

You'll get a report like the one below. It reports a list of all the blocks, their replication factor, and the set of hosts that the blocks are located on. 

/hdfs/path/to/data/file.txt 4771082824 bytes, 36 block(s):  OK
0. BP-22525430-10.14.103.78-1355873316066:blk_-3400885615428218530_203522 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
1. BP-22525430-10.14.103.78-1355873316066:blk_124203196739652236_203523 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
2. BP-22525430-10.14.103.78-1355873316066:blk_5886188080028552249_203524 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
3. BP-22525430-10.14.103.78-1355873316066:blk_-3222807870390148132_203525 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
4. BP-22525430-10.14.103.78-1355873316066:blk_-1285830390698132620_203526 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
5. BP-22525430-10.14.103.78-1355873316066:blk_-2680874809037637827_203527 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
6. BP-22525430-10.14.103.78-1355873316066:blk_8699277646297360652_203528 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
7. BP-22525430-10.14.103.78-1355873316066:blk_-2195916588803548138_203529 len=134217728 repl=3 [10.14.103.213:50010, 10.14.102.190:50010, 10.14.102.176:50010]
[more]

Upvotes: 1

rozar
rozar

Reputation: 1078

You can use localhost:50070/ to check the health of the name node. Using which you can browse the hadoop file system.

There is also few other things you can do localhost:50030 - JobTrack localhost:50060 - TaskTracker

Upvotes: 1

user1188611
user1188611

Reputation: 955

You can make use of web interface option available in hadoop. Using that you will be able to see every information about hdfs, how many nodes are present, how many are dead/alive, storage usage and all such. Hope it helps

Upvotes: 1

Related Questions