What is the kafka recommended configuration on “replication.factor” and “min.insync.replicas” with 6 brokers and acks=1?

Question

If I have a cluster with 6 brokers and on my producer configuration I’ve set "acks" to "1": What is the recommended configuration on "replication.factor" and "min.insync.replicas"?

I’ve thought: replication.factor: "4" min.insync.replicas: "1" Is correct this configuration? Can I have issues with this configuration?

I have 4 millions messages per hour, and I want to achieve load balancing and scalability, 6 brokers is better than 3 brokers for load balancing? What do you recommend me to do?

Answer 1

You should set the values in accordance to what you're willing to lose. If you set acks=1, and min.insync.replicas=1, then that one broker where the in-sync replica (of your 4) exists, then you will have downtime because an unclean (out of sync) replica can not be elected as leader, causing producer and consumer requests to start failing.

Other example, set acks=all, and min.insync.replicas=2 (also 4 replicas, but doesn't matter, just needs to be replication.factor >= min.insync.replicas), then you can lose up to 2 brokers before your cluster is degraded. If you keep acks=1, then you better ensure your replication speeds are faster than time-to-recovery of losing the leader broker that acked the request.

You've made no mention of those 4-million events needing to be guaranteed, so you could also just set acks=-1 and fire-and-forget. If you want guaranteed events, you'll will also want to look at the producer retries and enable.idempotence settings.

Two brokers is fine for load-balancing. 3 is better. 6 will give room to grow; it doesn't have to be an even number. If your topics have less than 6 partitions, then fully balanced load isn't really taken into account.