CODEWITHSUNDEEP
multithreadingcelerymessage-queueworker
Jaden
Jaden

Reputation: 322

When Celery will start a new worker?

I have one task for databases query. Earlier the worker2 got logged from our system. About 30 mins later, worker1 got logged for the same task, pretty much like restarting the job, which broke my sql query. My question is

From my code, I didnot set any time limit? Thank you in advance for any thoughts.

I am using job = chain(single_job), but i only have one single_job job() starting the job.

-Updates:

I solved this bug, if you have the same issue, you can do below.

Please check the visibility timeout for your celery

Please check the version to upgraded one.

Please cancel the retry for sql query, sometime sql query do not need to have message queue.

Upvotes: 0

Views: 144

Answers (1)

2ps
2ps

Reputation: 15946

There are about eighteen different scenarios that could cause what you describe. The three most common:

  1. someone scheduled your job to run on a schedule that occurred at both times
  2. someone / something else kicked off another instance of your task
  3. the task call failed somewhere which tirggered a retry that ran about 30 minutes later.

Upvotes: 1

Related Questions