CODEWITHSUNDEEP
pythonflaskrabbitmqcelery
Claudia
Claudia

Reputation: 59

Celery worker stops after being idle for a few hours

I have a Flask app that uses WSGI. For a few tasks I'm planning to use Celery with RabbitMQ. But as the title says, I am facing an issue where the Celery tasks run for a few minutes and then after a long time of inactivity it just dies off.

Celery config:

CELERY_BROKER_URL='amqp://guest:guest@localhost:5672//'
BROKER_HEARTBEAT = 10 
BROKER_HEARTBEAT_CHECKRATE = 2.0
BROKER_POOL_LIMIT = None

From this question, I added BROKER_HEARTBEAT and BROKER_HEARTBEAT_CHECKRATE.

I run the worker inside the venv with celery -A acmeapp.celery worker & to run it in the background. And while checking the status, for the first few minutes, it shows that one node is online and gives an OK response. But after a few hours of the app being idle, when I check the Celery status, it shows Error: No nodes replied within time constraint..

I am new to Celery and I don't know what to do now.

Upvotes: 1

Views: 6625

Answers (1)

Ángel Jiménez
Ángel Jiménez

Reputation: 180

Your Celery worker might be trying to reconnect to the app until it reaches the retry limit. If that is the case, setting up this options in your config file will fix that problem.

BROKER_CONNECTION_RETRY = True
BROKER_CONNECTION_MAX_RETRIES = 0

The first line will make it retry whenever it fails, and the second one will disable the retry limit.

If that solution does not suit you enough, you can also try a high timeout (specified in seconds) for your app using this option:

BROKER_CONNECTION_TIMEOUT = 120

Hope it helps!

Upvotes: 4

Related Questions