CODEWITHSUNDEEP
hdfscloudera-cdh
plaidshirt
plaidshirt

Reputation: 5681

Cloudera Manager - HDFS Under-Replicated Blocks

I use a fresh install of CDH 5.5.2. In Cloudera Manager I see, that HDFS has a critical health issue. I have 1 NameNode and 1 DataNode in cluster.

Under-Replicated Blocks

Under-Replicated Blocks

578 under replicated blocks in the cluster. 580 total blocks in the cluster. Percentage under replicated blocks: 99.66%. Critical threshold: 40.00%

How should I fix this issue?

UPDATE: Cloudera Manager(CDH 5.0.2) -> HDFS -> Configuration -> View and Edit -> Service-Wide -> Replication -> Replication Factor (dfs.replication) -> 1 isn't working in this case.

Upvotes: 1

Views: 3564

Answers (2)

Alexandr Eroshenko
Alexandr Eroshenko

Reputation: 91

The new replication factor affects only new files. To change replication factor for existing files run in shell (on the node with hadoop entry point)

hadoop fs -setrep -w <replication factor> -R /

But, only "hdfs" can write to / ("hdfs" is the superuser, not "root"). So, may be you will have to use this command:

sudo -u hdfs hadoop fs -setrep -w <replication factor> -R /

Upvotes: 2

Okezie
Okezie

Reputation: 5118

A few things to check:

  • The namenode may be running with outdated configuration. If so, restart the namenode to make sure that it is running with the current configuration displayed in cloudera manager.
  • Sometimes the cloudera agent needs to be restarted as well for it to generate configurations properly
  • If that doesn't work, then run hadoop fsck /|egrep -v '^\.+$'|grep -i replica
  • find out the files affected and verify that the replication factor is set to 1 since you have only one datanode
    • you can check replication factor for a file by running hadoop fs -ls /path/to/file.txt then look at the number in the 2nd column in the output

Note: be sure to add the HDFS Gateway role to the server you want to run the hadoop commands from.

Also consider upgrading cloudera manager to 5.5.1 - your version is a very early version and they made a TON of changes since then.

Upvotes: 1

Related Questions