CODEWITHSUNDEEP
redis
rococo
rococo

Reputation: 2657

Check the current position in a Redis list of some list element

I have a simple job queue on Redis where new jobs are pushed with RPUSH and consumed with BLPOP. The jobs are stringified JSON objects that have an id field among other things (the json string is parsed by the workers).

Each job takes some time to do, so there can be a meaningful wait time. I'd like to be able to find a job's current position in the queue, so that I can give an update to whatever is waiting on that job. That is, be able to do something like "your current position is 300... 250... 200... 100... 10... your job is now being processed".

It can be assumed that the list may grow long but never too long, i.e. possibly 1000 entries but not 1 million.

After looking through the docs a bit, it seems like this is maybe easier said than done. A possible naive solution seems to be to just loop through the list until the element is found. Are there any performance issues with calling LINDEX a couple hundred times at a time like that?

Would appreciate any suggestions on other ways this can be done (or confirmation that LINDEX is the only way). The whole structure (even the usage of a list, or addition of some helper map/list) can be changed if needed, only requirement is that it run on Redis.

Upvotes: 0

Views: 368

Answers (1)

for_stack
for_stack

Reputation: 22936

You can use a sorted set and a counter to more elegantly solve the problem.

Push a job

  1. Call INCR counter to get a counter.
  2. Use the counter as score of the job, and call ZADD jobs counter job-name

Pop a job

Call BZPOPMIN jobs to get the first unprocessed job.

Get job position

Call ZRANK jobs job-name to get the rank of the job, e.g. the current position of the job.

Upvotes: 2

Related Questions