CODEWITHSUNDEEP
cachingredis
randombits
randombits

Reputation: 48450

Redis: possible to expire an element in an array or sorted set?

Is it currently only possible to expire an entire key/value pair? What if I want to add values to a List type structure and have them get auto removed 1 hour after insertion. Is that currently possible, or would it require running a cron job to do the purging manually?

Upvotes: 167

Views: 80796

Answers (5)

Jonathan Oren
Jonathan Oren

Reputation: 47

Since Redis 7.4 there's the command HEXPIRE which should be sufficient for most use-cases out-of-the-box.

https://redis.io/docs/latest/commands/hexpire/

As of writing this Redis 7.4 is only available in their Enterprise Software.

EDIT: 7.4 is now available in the Community Edition, implementation will generally look like this:

  1. Store the info as key:value in hash: HSET customer:1 first_name "Bob" last_order "pizza"
  2. Add key to list (LPUSH items key) or if you need scoring, to a sorted set (ZADD items score key).
  3. Set expiration: HEXPIRE key 3600 FIELDS 1 field.
  4. When getting data from list/sorted set, if key doesn't exist in hash you can safely remove it.

Example:

import redis
from time import time

r = redis.Redis(host='localhost', port=6379, decode_responses=True)

# Better to use a transaction
with r.pipeline() as pipe:
   # Store customer data in hash
   pipe.hset("customer:1", mapping={
       "first_name": "Bob", 
       "last_order": "pizza"
   })

   # Add to list (or sorted set)
   pipe.lpush("customers", "customer:1")
   # pipe.zadd("customers", {"customer:1": int(time())})  # If using sorted set

   # Set fields to expire in 1 hour
   pipe.hexpire("customer:1", 3600, "first_name", "last_order")
   
   pipe.execute()

# Later when getting customers...
customers = r.lrange("customers", 0, -1)  # For list
# customers = r.zrange("customers", 0, -1)  # For sorted set

for cust_key in customers:
   data = r.hgetall(cust_key)
   if not data:  # Hash expired
       r.lrem("customers", 0, cust_key)  # Remove from list
       # r.zrem("customers", cust_key)  # If using sorted set
   else:
       print(f"Customer: {data}")

Upvotes: 2

Fish
Fish

Reputation: 39

What about creating two seperate sorted sets?

Main sorted set which is key = value.

Expire sorted set which is key = expire_timestamp. If you only want to expire a single score you can set as key:unique_id = expire_timestamp.

With mercy of zrangebyscore we can get expired keys. Then all we need to do is check periodically and zrem. If you only want to expire a single score: ​zincrby -1.

Upvotes: -1

JEPrice
JEPrice

Reputation: 693

I came upon a different method of handling this, don't know if it's helpful to any of you,but here goes:

The hash and the sorted set are linked by a guid.

  1. I have a hash that is set to expire in 'x' seconds
  2. I have a sorted set that is used for ranged queries
  3. The data for both is added in a transaction, so if one fails, they both fail.
  4. Upon a ranged query, use 'EXISTS' to see if the hashed value exists as the results are iterated over
  5. If it does not exist, it has expired, so delete the item from the sorted set

Upvotes: 5

yojimbo87
yojimbo87

Reputation: 68305

Is it currently only possible to expire an entire key/value pair?

As far as I know, and also according to key commands and document about expiration, currently you can set expiration only to specific key and not to it's underlying data structure. However there is a discussion on google groups about this functionality with outlined alternative solutions.

Upvotes: 85

Adriaan Pelzer
Adriaan Pelzer

Reputation: 1489

There is a common pattern that solves this problem quite well.

Use sorted sets, and use a timestamp as the score. It's then trivial to delete items by score range, which could be done periodically, or only on every write, with reads always ignoring the out of range elements, by reading only a range of scores.

More here: https://groups.google.com/forum/#!topic/redis-db/rXXMCLNkNSs

Upvotes: 143

Related Questions