CODEWITHSUNDEEP
apache-kafkaapache-flink
Urjit Patel
Urjit Patel

Reputation: 41

Topic side compression in apache kafka

If we do not mention the compression type on kafka producer, and if we mention it on the broker side... How is the performance impacted and in what batch sizes does topic side compression work on?

Upvotes: 0

Views: 501

Answers (1)

Narayan Periwal
Narayan Periwal

Reputation: 99

Compression will work only if you specify on the producer side, otherwise data will be stored in uncompressed format on the disk. Compression increases the I/O throughput for some compression and decompression cost on the client side. Also, it saves disk space as data will be stored in the compressed format in the kafka brokers. You can keep the batch size up to the maximum message size allowable limit by the kafka broker, that is 1 MB

Upvotes: 1

Related Questions