CODEWITHSUNDEEP
apache-kafka
Learner
Learner

Reputation: 45

kafka cluter not all replication are coming as ISR

We have multinode Kafka 1.0.1 cluster (tried earlier version also) and when we are creating this cluster we have given replication as n-1 so if 7 we have nodes we are giving 6 as replication factor but not all replicas are becoming ISR. This issue happens for some topics and for some it create. If we recreate multinode cluster 2-3 times it creates ISR for all replicas. Did someone got similar issue and help on this greatly appreciated.

Upvotes: 0

Views: 151

Answers (1)

Matthias J. Sax
Matthias J. Sax

Reputation: 62310

  1. Your replication factor seems to be quite high. Why do you set it to n-1? The replication factor should be independent from the cluster size. For most use cases a replication factor of 3 is sufficient. There are also use cases with stronger demands that use a replication factor of 5. I guess a higher replication factor would be very rare. If you don't have special needs, 3 should be sufficient.

  2. The larger the replication factor, the more time it needs to replicate data -- thus, if you continuously write to the leader, the followers need some time to copy the data, and the network bandwidth to copy the data is obviously limited. Thus, with a larger replication factor, each individual follower has less bandwidth for copying the data and might fall back and thus never become an ISR.

Upvotes: 2

Related Questions