CODEWITHSUNDEEP
apache-kafka
user13335774
user13335774

Reputation: 1

Will Kafka retention will also partition Replicas also?

In process of estimating storage needs for 3 node kafka cluster,when we define retention of 3days on topic with 1 partition having 3 replicas assuming streaming 1GB/day will it need 1GB x 3 Replicas and retention for 3 days 9 GB across 3 brokers.. or will it retention only 1 partition data for 3 days ie 3GB overall across all the brokers?

Upvotes: 0

Views: 48

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191728

Replicas are for each message, so if you have 1GB of messages hitting one partition of a topic with replication.factor=3, then that amounts to a total of 3GB

Note: There are internal topics like _consumer_offsets as well that should be accounted for (its retention and replicaton factor).

Upvotes: 1

Related Questions