CODEWITHSUNDEEP
pythondjangoceleryflower
grrrrrr
grrrrrr

Reputation: 1425

Is celery following my rate limit properly?

I have a rate-limited task in Celery but the results as shown in Flower make it appear as though it is processing a bunch of tasks right away and then obeying the rate limit. Why is this happening? Is there something I need to do to make sure that it obeys the rate limit from the start?

@shared_task(rate_limit="4/m")
    def my_task(a,b):
        ...

...

my_task.apply_async((x, y,),)

succeeded tasks

steps

Upvotes: 0

Views: 1405

Answers (1)

John Lucas
John Lucas

Reputation: 648

Do you have more than one worker consuming my_task? If so your rate limit will not be taken into effect since celery rate limits are per worker (docs). To get around this you'll need to route my_task to a queue that has only one worker consuming from it. See routing docs.

Upvotes: 5

Related Questions