Banupriya
Banupriya

Reputation: 195

How is flink rocks DB live data size and checkpoint size is related?

I have flink pipeline with RMQ source, filters, keyby, enrichers, map, aggregator and tumbling window. Rockdb incremental checkpoints is enabled.

I have enabled rocks db metrics to monitor my growing checkpoint size.

when my check point size is 1mb, live data (state.backend.rocksdb.metrics.estimate-live-data-size) in rocks db is just 39kb.

Does that mean 916kb(1mb-39kb) is stale data and will it be removed during next compaction?

Upvotes: 0

Views: 78

Answers (0)

Related Questions