CODEWITHSUNDEEP
pythoncelery
Overdrivr
Overdrivr

Reputation: 6596

How to run only selected tasks in celery

I have two celery tasks, implemented in two different files. Task1 needs to launch task2, so task2 is imported from task1.

mypkg
 | tasks
    | task1.py
    | task2.py

task1.py

from mypkg.tasks.task2 import task2

celery_app = Celery('mypkg.tasks.task1', broker='redis://localhost')

@celery_app.task
def task1():
   print('Do task 1')
   // Then call task2
   task2.delay()

task2.py

celery_app = Celery('mypkg.tasks.task2', broker='redis://localhost')

@celery_app.task
def task2():
   print('Do task 2')

When I launch celery with the command below, both tasks run although I was expecting only task1:

pipenv run celery -A mypkg.tasks.task1 worker --loglevel=info

Most likely this is because task1 imports task2. How can I tell celery to only run tasks found in task1 ?

Upvotes: 0

Views: 2068

Answers (1)

2ps
2ps

Reputation: 15946

If this is what you really want, just sit your instances on different broker urls, e.g.,

celery_app = Celery('mypkg.tasks.task1', broker='redis://localhost/1')

. . . and . . .

celery_app = Celery('mypkg.tasks.task2', broker='redis://localhost/2')

While you can do this, I would recommend instead using routing and queues to specify which tasks are handled by which workers. Routing is an in-depth topic, but once you get the hang of it, it's fairly straightforward.

Upvotes: 2

Related Questions