Reputation: 195
I have flink pipeline with RMQ source, filters, keyby, enrichers, map, aggregator and tumbling window. Rockdb incremental checkpoints is enabled.
I have enabled rocks db metrics to monitor my growing checkpoint size.
when my check point size is 1mb, live data (state.backend.rocksdb.metrics.estimate-live-data-size
) in rocks db is just 39kb.
Does that mean 916kb(1mb-39kb) is stale data and will it be removed during next compaction?
Upvotes: 0
Views: 78