CODEWITHSUNDEEP
performancerediscustomizationredis-cluster
Mahdi Yusefi
Mahdi Yusefi

Reputation: 521

Customize Redis Expiration Keys Algorithm

I have a performance issue in one of my microservice project, that is high latency in connecting or performing command to redis. After some investigation, I came into this:

According to Redis documentation, redis algorithm for expiring keys is:

  1. Test 20 random keys from the set of keys with an associated expire.
  2. Delete all the keys found expired.
  3. If more than 25% of keys were expired, start again from step 1.

In my project, 30 keys per second are created or updated, which has a ttl of 10 minutes, that is about one third of all stored keys.

Does this mean in most execution of an algorithm we fulfil the condition of third step and redis consume most of its resource on expiring key? If this so, can I customize or override default behaviour of this algorithm by changing some config? Or is there any alternative solution?

Upvotes: 0

Views: 1709

Answers (1)

garrett Kocher
garrett Kocher

Reputation: 141

If your additional keys are non-volitile then they shouldn't be in the consideration of 25% or more being expired.

The default redis config has active-expire-effort 1 on the range of 1-10. Increase would only allocate more CPU to cleanup. However, increasing may also not have an effect if the expiry isn't the root cause for your latency. From the information here, it seems key expiry might not be the issue given ~18k volatile keys you would have.

In terms of alternatives, you might consider further troubleshooting.

Upvotes: 1

Related Questions