CODEWITHSUNDEEP
pythonasynchronouscelery
Tarjintor
Tarjintor

Reputation: 645

how to callback in celery but not block the process

Now I have some big tasks, and all of them made of some small tasks,I put them into a broker,and when all small tasks of one big task is finished,I will need a callback to deal with the results of these small tasks I know celery have the primitive chord can do it if only I have one big task,but I have many so,if I write like:

chord([task11.s(),task12.s()])(mycallback.s()).get()
chord([task21.s(),task22.s()])(mycallback.s()).get()

the second line won't start until the first line is finished,but in this way,some workers would be idol for a long time,which is not good, so is there some in way in celery that can callback when certain condition happens,but not block the process?

==============================================

apply_async works!@Gigapalmer, thank you for help

Upvotes: 1

Views: 705

Answers (1)

chaos
chaos

Reputation: 490

Try linking the async methods so that when one finishes you call the next one in chain with apply_async and the required parameters that will technically create a new job (should be on a different queue).

Upvotes: 1

Related Questions