CODEWITHSUNDEEP
stateapache-flink
liaoyue
liaoyue

Reputation: 40

Why isn't there a size method for flink MapState?

I need to count daily orders per user. The data may arrive late for quite a long time. So i think i can use a MapState[String,Long] for key is date and value is the order num according to the date to store the count.But as time goes by, the key-value will be one more per key per day and the state size could be too big someday. Since the data won't be late for longer than a day ,i just need to store two days of data. In this situation, i need to remove the earliest date when the size of MapState[String,Int] reaches 3. But i find out that there isn't a size method for fink MapState.

I know i can use iterator to achieve this and this is exactly what i did. But since the java.util.Map has size method, why isn't there a size method for flink MapState?

Upvotes: 1

Views: 751

Answers (2)

Xiaowei Tan
Xiaowei Tan

Reputation: 36

check out this: https://issues.apache.org/jira/browse/FLINK-5917 and it explains the reason. size() has been removed since then.

Upvotes: 2

David Anderson
David Anderson

Reputation: 43419

This is because with the RocksDB state backend the underlying representation of MapState doesn't allow for an efficient implementation of a size method.

Upvotes: 1

Related Questions