CODEWITHSUNDEEP
pythoncelery
uchiha itachi
uchiha itachi

Reputation: 195

Run a same celery task in loop

how to run this kind of celery task properly?

@app.task
def add(x)
    x + 1


def some_func():
    result = 'result'
    for i in range(10):
        task_id = uuid()
        add.apply_async((i,)), task_id=task_id)
    return result

I need all tasks to be performed sequentially after the previous one is completed. I tried using time.sleep() but in this case returning result waits until all tasks are completed. But I need the result returned and all 10 tasks are running sequentially in the background.

there is a group() in celery, but it runs tasks in parallel

Upvotes: 2

Views: 2228

Answers (2)

uchiha itachi
uchiha itachi

Reputation: 195

Finally, I solved it by using immutable signature and chain

tasks = [
    add.si(x).set(task_id=uuid())
    for x in range(10)
]
chain(*tasks).apply_async()

Upvotes: 2

DejanLekic
DejanLekic

Reputation: 19822

If some_func() is executed outside Celery (say a script is used as "producer" to just send those tasks to be executed), then nothing stops you from calling .get() on AsyncResult to wait for task to finish, and loop that as much as you like.

If, however, you want to execute that loop as some sort of Celery workflow, then you have to build a Chain and use it.

Upvotes: 2

Related Questions