Scenario: I have a big list of products which have the following properties These product availability/price (origin data) are cached and will be used by multiple clients. Each client will have different markup rule for them and therefore different final prices. I need to apply filtering on those products based on price, etc.. However, loading them from redis and re-applying rules to build filter with redis is expensive. Is there a better way to do this?

Reputation: 7237

Filtering large set with Redis

Scenario:

I have a big list of products which have the following properties

These product availability/price (origin data) are cached and will be used by multiple clients.
Each client will have different markup rule for them and therefore different final prices.

I need to apply filtering on those products based on price, etc..

However, loading them from redis and re-applying rules to build filter with redis is expensive.

Is there a better way to do this?

Upvotes: 1

Reputation: 3063

This is feasible to some degree by using sortedsets (Z* commands) lexical ordering & compound sorted-set member keys assuming that:

Filtering criteria will be part of the sorted-set member keys
Filtering criteria are finite & deterministic (all of them appear all of every member in the list, or at least it can be represented that way by using some sort of place holders like the keyword :na: where criteria is not present )
Filtering criteria values are scalar in nature, and do not represent a range of data. Note that this is not mandatory, but if you still choose too represent data that has to much 'ranged' value keys, you'll wind up with 'Cartesian products' causing repository maintenance(writes/updates) to be a nightmare and creating way, way too many keys for your redis instance or even cluster to handle.

If all of the above apply, SCAN (or ZSCAN to be more specific), is your friend

You can also find more information here

Upvotes: 2