CODEWITHSUNDEEP
rediskeykey-value
CodingInCircles
CodingInCircles

Reputation: 2807

How to access multiple, different keys in production using Redis?

I have an application for which I'm storing millions of keys in Redis in the format:

Type+#+Year+#+MachineType+#+City+#+State+#+Country+#+Size

Sample_Key                                   Value

Retail#2017#MachineA#SanFrancisco#CA#USA#500 1000
Bulk#2017#MachineB#NewYorkCity#NY#USA#1000   100000
Retail#2017#MachineA#NewYorkCity#NY#USA#1000 5000

My customers would come in and want a specific value or set of values aggregated, so say, everything in San Francisco, CA and New York, NY, so as to get 1000 and 5000 (and then we'd perform some aggregation on it). What Redis function(s) are best suited to say, "Give me all values for keys that contain either San Francisco or New York", or "Give me all keys for store sizes of 1000", or any such combination.

I'd need to get all these values, aggregate them and serve it out with (ideally) millisecond-level latency.

I have looked up KEYS which they say NOT to use in Production (Plus, I never quite figured out how to get the values for BOTH (or potentially more) cities/other combinations). I have looked up SCAN, but being cursor-based, it might not be the best solution. There's nothing else that I have looked at which quite covers the scenario I've described in a quick and easy way. Help?

Upvotes: 0

Views: 493

Answers (1)

kelgon
kelgon

Reputation: 401

You are right about don't use KEYS. In your case SCAN is bad too, because you have to iterate over a large amount of keys(millions). SCAN won't block other commands for too long, but the total time cost won't shorten. If you run SCAN command often, the load on Redis will be quite heavy. In fact I think Redis doesn't fit for this case, RDBMS is better.

I have 2 suggestions:

  1. Use SCAN and be aware of the potential risk, do some serious test.
  2. Store and maintain a key relationship(e.g. City-Keys) somewhere, first find the keys you need, then get the value of these keys from redis. In your case, I'd suggest put all keys in a RDBMS table, and use LIKE to find the ones you need(use search engine such as solr or elasticsearch is much better on performance, of course)

Upvotes: 1

Related Questions