CODEWITHSUNDEEP
hadoophdfs
user2633667
user2633667

Reputation: 1

HDFS over and under replication

When does over replication occur in HDFS? Since the replication factor is set in the hdfs-site.xml and the replication is taken care by the Hadoop framework.

Upvotes: 0

Views: 410

Answers (1)

PradeepKumbhar
PradeepKumbhar

Reputation: 3421

About under replication: This happens when one or more Datanodes in the cluster go down because of some failure and never come back. In such cases, the expected replication factor for blocks stored on those down nodes decreases.

About over replication: This happens again when one or more Datanodes go down but come back after a while. For that short downtime duration, Namenode sees that some blocks are under-replicated and starts replication of under-replicated blocks to bring the block count to normal. If after some time those down Datanodes come back up again, then there will be extra copies of the blocks than expected.

Upvotes: 1

Related Questions