CODEWITHSUNDEEP
databaseredis
dzm
dzm

Reputation: 23554

Limit list length in redis

I'm using redis lists and pushing to new items to a list. The problem is I really only need the most recent 10 items in a list.

I'm using lpush to add items to a list and lrange to get the most recent 10.

Is there anyway to drop items after a certain number? I'll end up with lists that may have 1,000's of items and can cause performance issues with latency.

Thank you!

Upvotes: 48

Views: 44357

Answers (6)

giridhar
giridhar

Reputation: 151

Calling LTRIM <list-name> -1 -10 after LPUSH <list-name> <item> is the simplest answer. Many had covered it.

You must do this two operations in a transaction or must use Lua script to ensure the operation is atomic.

Upvotes: 0

kuzey beytar
kuzey beytar

Reputation: 3226

No one has ever mentioned the real solution about storing only most 10 recent items.

Let's create a sample list with 15 items (here just numbers):

RPUSH list 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

Now indicate offset from the end of the list:

LTRIM list -10 -1

Show list

LRANGE list 0 -1

 1) "6"
 2) "7"
 3) "8"
 4) "9"
 5) "10"
 6) "11"
 7) "12"
 8) "13"
 9) "14"
10) "15"

Now you can add new items and run trim:

RPUSH list 16
LTRIM list -10 -1

 1) "7"
 2) "8"
 3) "9"
 4) "10"
 5) "11"
 6) "12"
 7) "13"
 8) "14"
 9) "15"
10) "16"

Upvotes: 12

wei tu
wei tu

Reputation: 119

Just an alternative. According to official doc of LPUSH, it returns the length of the list after the push operations. You can set a threshold length like k (in your case k > 10) and call LTRIM when returned length is bigger than k. Sample pseudo code as follows:

len = LPUSH mylist xxx 
if len > k:   
  LTRIM mylist 0 9
LRANGE mylist 0 9

It's more controllable than random method. Greater k triggers less LTRIM but with more memory cost. You can adjust k according to how often you want to call LTRIM since calling extra command is more expensive.

Upvotes: 10

DhruvPathak
DhruvPathak

Reputation: 43245

You can use LTRIM intermittently after any LPUSH, no need to call LTRIM after every LPUSH as that would add to overall latency in your app ( though redis is really fast, but you can save lots of LPUSH operations )

Here is a pseudo code to achieve an LTRIM on approximately every 5th LPUSH:

LPUSH mylist 1
random_int = some random number between 1-5
if random_int == 1:  # trim my list with 1/5 chance
   LTRIM mylist 0 10

Though your list may grow to be a few elements more than 10 elements at times, but it will surely get truncated at regular intervals. This approach is good for most practical purposes and saves a lot of LTRIM operations, keeping your pushes fast.

Upvotes: 42

ovunccetin
ovunccetin

Reputation: 8663

The following code,

  • pushes the item to the list,
  • keep the size fixed to 10,
  • and returns the most recent 10 elements

in a transaction.

MULTI
LPUSH list "item1"
LTRIM list 0 9
LRANGE list 0 9
EXEC

Upvotes: 12

Sripathi Krishnan
Sripathi Krishnan

Reputation: 31528

After every lpush, call ltrim to trim the list to 10 elements

See http://redis.io/commands/ltrim

Upvotes: 61

Related Questions