CODEWITHSUNDEEP
javascalahdfsapache-flinkflink-streaming
juanmirocks
juanmirocks

Reputation: 6239

Flink: possible to delete Queryable state after X time?

In my case, I use Flink's queryable state only. In particular, I do not care about checkpoints.

Upon an event, I query the queryable state only after a maximum of X minutes. Ideally, I would delete the "old" state to save on space.

That's why I wonder: can I signal Flink's state to clear itself after some time? Through configuration? Through specific event signals? How?

Upvotes: 1

Views: 672

Answers (1)

David Anderson
David Anderson

Reputation: 43697

One way to clear state is to explicitly call clear() on the state object (e.g., a ValueState object) when you no longer need it for a particular key. This is typically done in an onTimer() callback in a ProcessFunction.

Another possible approach would be to use state time-to-live to manage its lifecycle.

I haven't tried using state TTL with queryable state, but I can't see any reason why it shouldn't work. However, as of Flink 1.7, state TTL only actually clears state (for a key) when the state is accessed (for that key), or when taking a full state snapshot. So in your particular case, this state TTL mechanism may not be very useful.

Upvotes: 2

Related Questions