CODEWITHSUNDEEP
performanceredis
Eit
Eit

Reputation: 41

Redis Stream Managing a time frame

I am writing events into Redis Stream. But I would like to keep only the last 48 hours events. According to the Redis documentations, I saw that I can manage my list size only using the MAXLEN which take affect by the records count and not by time frame. Is there any way I can use the XADD function but to limit on insertion records oldest that the last 48 hours? Thanks for the help!

Upvotes: 4

Views: 1091

Answers (3)

Jake Lowen
Jake Lowen

Reputation: 929

A possible solution in nodejs based on trimming to a specified key (not on time per se).

https://gist.github.com/jakelowen/22cb8a233ac0cdbb8e77808e17e0e1fc

Proof of concept. Not battle tested.

Upvotes: 0

antirez
antirez

Reputation: 18514

This is yet not clear. I don't like the vanilla way of time capping a stream, that is, "trim by <seconds>", because it means that if there is a delay in the process XADD-ing items, later the next XADD will have to evict things potentially for seconds, causing latency spikes. Moreover it does not make a lot of sense semantically. Your real "capped resource" is memory, so it's not really so important how many items you want to store in the past VS how many items you can store, so the number of items limit makes more sense. Yet in certain applications where there are multiple streams with insertion rates that vary a lot between different producers, it makes sense to cap by time, to avoid wasting memory in certain producers that emit very few entries per unit of time. Maybe at some point I'll add some "best effort" time capping that does not do more work than a given amount, but that eventually will be able to trim the stream, given enough XADD calls.

Upvotes: 5

Itamar Haber
Itamar Haber

Reputation: 50112

AFAIK not yet. There were discussions about adding a timestamp cap (to XADD, and possible to XTRIM as well), but it doesn't look like this feature has been implemented in the latest release candidates.

Upvotes: 1

Related Questions