CODEWITHSUNDEEP
pythonredisrabbitmqcelerycelery-task
William Afonso
William Afonso

Reputation: 96

How to make Celery waiting for a specific delay before executing a new task

I am using Celery to parallelize the execution of a Python function calling a third-party API.

This API imposes to wait for at least 3 seconds between each call.

Is there a way to specify a Message Broker (RabbitMQ or Redis) to respect this delay between each worker call ?

Upvotes: 0

Views: 6027

Answers (2)

jjdoherty
jjdoherty

Reputation: 504

Would check out https://docs.celeryproject.org/en/stable/userguide/tasks.html#retrying

Specify the retry delay to be 3 seconds and you can set a limit for the number of retries to be X as part of the task definition as seen in the docs.

Upvotes: 0

Chad Knutson
Chad Knutson

Reputation: 391

In Celery, you can use the countdown method. See https://docs.celeryproject.org/en/stable/userguide/calling.html#eta-and-countdown

RabbitMQ has a few different options for supporting delayed messages including the delayed message exchange plugin and dead lettering. Unfortunately, Celery doesn't support either of these methods. What it does instead is delay execution of the message until the countdown time is reached. The message itself is immediately sent to the worker.

Upvotes: 1

Related Questions