CODEWITHSUNDEEP
rabbitmqcelery
Bialecki
Bialecki

Reputation: 31071

When does a celery worker acknowledge to RabbitMQ that it has a task?

I might be misunderstanding how this works (which is why I'm asking), but I think when a celery worker consumes a task from RabbitMQ it puts a lock on it -- so to speak -- and then must acknowledge it completed that task onces it's done. So say I have 4 workers which all have the prefetch setting at 1 and queue of 6 tasks which take a long time. Once I start those workers and I run:

rabbitmqctl -q list_queues name messages messages_ready messages_unacknowledged

I'd expect to see something like:

celery  6       2       4

indicating that 4 tasks are running (but not yet acknowledged) and 2 are ready to be consumed.

I think my understanding is wrong because what I actually see is:

celery  2       0       2

So it's as if the acknowledging happens when a message is received by a worker, but before that worker finishes processing that task.

So to sum up, my question is, when does a celery worker acknowledge it has a task? It seems like it's once it receives that task and starts working on it, not when it completes working on it. Can someone confirm?

Upvotes: 21

Views: 13743

Answers (1)

asksol
asksol

Reputation: 19479

This is mentioned in the FAQ, but I can't blame you for not finding it: http://docs.celeryproject.org/en/latest/faq.html#should-i-use-retry-or-acks-late

The default behavior of early ack is there because we don't want to enforce users to write idempotent tasks.

Upvotes: 28

Related Questions