CODEWITHSUNDEEP
hdfs
icarus
icarus

Reputation: 309

Rebalance while decommissioning a datanode

I am currently decommissioning a datanode in my HDFS cluster. I noticed that some of the datanodes are very full, and so I was hoping to rebalance them out.

What happens when one performs a rebalance of an HDFS cluster while decommissioning a data-node? Is this a safe operation?

Upvotes: 1

Views: 660

Answers (1)

Abhinav
Abhinav

Reputation: 676

You don't need to perform the balancing operation. If your replication factor is greater than 2 then you already have replicated data on other nodes.

Balancing is a concept used when your data is replicated in unequal partitions. Suppose your data is has a replication of 70% on one node and 30% on other. Then there may be under-replicated blocks or over-replicated. To maintain this, we use balancing concept.

Talking about decommissioning your Datanodes, then it is always a safe operation. Because your Namenode is capable of replicating the data(That relies on the decommissioned Datanode) on other Datanodes in your cluster. You don't need to balance the data by yourself. Namenode will take care of it.

P.S:- Just make sure your other Datanodes have enough space to copy all the data onto them.

Upvotes: 1

Related Questions