CODEWITHSUNDEEP
pythonredisceleryceleryd
Maksym Polshcha
Maksym Polshcha

Reputation: 18358

Celery + Redis backend: How to limit queue size?

Is there a way to limit queue size when I run Celery with Redis backend?

Something like x-max-length in queue predeclare for rabbitmq

Upvotes: 4

Views: 3639

Answers (3)

luisgc93
luisgc93

Reputation: 184

This might be a bit hacky but you could try using a Redis lock when calling the task. That way if another process wants to call the task, it would have to wait for the Redis lock to be released (this would happen when either the task is done running or when the timeout is reached). This would prevent too many tasks being added to the queue

 with r.lock(some_lock_name, blocking_timeout=10):
    your_celery_task.delay()

Upvotes: 0

DejanLekic
DejanLekic

Reputation: 19787

As far as I know that is not possible with Redis as backend.

Upvotes: 1

Bharat Gera
Bharat Gera

Reputation: 820

I think you are looking for prefetch limits in celery. Check it out docs.

Upvotes: 0

Related Questions