CODEWITHSUNDEEP
apache-kafka
wandermonk
wandermonk

Reputation: 7356

Manual flush of messages to disk in kafka

I would like to know if there is a better way to handle the flush rates of data from the cache to disk while the cluster is up and running apart from setting the flush interval.

Upvotes: 1

Views: 1525

Answers (1)

Mickael Maison
Mickael Maison

Reputation: 26885

No, Kafka only exposes flush interval configs, in seconds log.flush.interval.ms or number of messages log.flush.interval.messages.

The flush mechanism is detailled in a couple of sections in the Kafka docs:

Kafka actually recommends keeping explicit flush disabled (that's the default):

We recommend using the default flush settings which disable application fsync entirely. This means relying on the background flush done by the OS and Kafka's own background flush. This provides the best of all worlds for most uses: no knobs to tune, great throughput and latency, and full recovery guarantees. We generally feel that the guarantees provided by replication are stronger than sync to local disk, however the paranoid still may prefer having both and application level fsync policies are still supported.

Upvotes: 2

Related Questions