CODEWITHSUNDEEP
redis
Positonic
Positonic

Reputation: 9411

Scan a redis set for certains values and create and action in a queue

I believe using Redis is the best way to do what I need to do, but please correct me if I'm wrong.

I open a socket to an exchange and receive a steady stream of stock prices, about 1 per "trading pair" per second.

I have users that have set alerts in MYSQL to be notified when the price hits a certain price point.

My first naive idea would be to sort the alerts for a trading pair and when I receive a new price from the socket to loop through the list of alerts in mysql and notify any users that need to be notified.

This is clearly not going to work, as it will be a huge workload for poor MYSQL. So my idea is to use redis, and hold the alerts in Redis.

I'm not super experienced in Redis development though and I am struggling to find a solution. At first I need some sort of sorted list or hash, but I can only store 1 piece of information in a list...

I believe I need to hold 3 pieces of information the database :

1) The 'trading pair' (name of the list)

2) The price

3) The id of the watcher record in MYSQL so I can notify them

So I could create a map for each watcher

watcher_(watcherID)

like:

watcher_3

    price : 8000

Then create a set with the watcher ids like:

3,5,6

and call sort descending on that using "Sorting by external keys"

Then somehow loop over that using SCAN and MATCH, until I hit a price that is less than or equal the current price, and move it off to a new queue.

Perhaps sorting is not even necessary, if I can just iterate over the set asking the same question.

Even simply thinking this through using Redis is a real pain in my face, and I'm not even sure I'm going about it correctly.

Can anyone advise me if the approach above would work, or if there's a better approach?

Upvotes: 1

Views: 156

Answers (1)

Imaskar
Imaskar

Reputation: 2939

Looks like you could use a sorted set and a pub-sub. Use your pair name and lt/gt as a part of the key. Price would be item's score and watcher id would be item's data. Then when a quote comes you call ZRANGEBYSCORE to get which watcher should be called.

So, for example you have these watchers (>=100, >=150, <=200): 

127.0.0.1:6370> zadd watchers:usdeur:gt 100 1
(integer) 1
127.0.0.1:6370> zadd watchers:usdeur:gt 150 2
(integer) 1
127.0.0.1:6370> zadd watchers:usdeur:lt 200 3
(integer) 1

And comes a quote 'usdeur:130':

127.0.0.1:6370> ZRANGEBYSCORE watchers:usdeur:lt 130 +inf
1) "3"
127.0.0.1:6370> ZRANGEBYSCORE watchers:usdeur:gt -inf 130
1) "1"

Now you can process these watchers immediately, or send a message to pubsub channel to be processed by other workers.

Upvotes: 2

Related Questions