CODEWITHSUNDEEP
rediscelerytask
vin
vin

Reputation: 1019

Acks_late: celery + redis broker / backend

I was going through celery code. Acks_late is called once the task function runs via (task_trace). However, in Redis, the once a task is received (i.e pop from Redis Queue) RedisWorkerController creates a task request for it. How is it enqueued again in the event the worker node dies?

Upvotes: 2

Views: 2632

Answers (2)

Mike Trotta
Mike Trotta

Reputation: 686

For more details on the implementation specifics of this mechanism, you can check out the "Redis Ack Emulation" part of the redis kombu transport.

From this code, we can see that celery implements ack emulation in Redis by adding the message to a sorted set prior to dequeuing from the main task queue, preventing message loss in the event of a worker failure. This allows celery to detect and re-queue tasks that have not been ACK'ed by their visibility timeout.

Upvotes: 0

suligap
suligap

Reputation: 41

The messages aren't enqueued again in case of them not being acknowledged (It would be impossible if the worker dies. They do exist in Redis as unacknowledged).

According to celery docs, Redis broker has a visibility timeout mechanism.

So we should be able to expect the message to be delivered again to a worker if it was not acknowledged within the visibility timeout. And that's what happens. If the power goes out during the processing of an acks_late task, the task is received again by an online worker after the visibility timeout is passed.

Upvotes: 4

Related Questions