Celery tasks functions - web server vs remote server

Question

I'm willing to send tasks from a web server (running Django) to a remote machine that is holding a Rabbitmq server and some workers that I implemented with Celery.

If I follow the Celery way to go, it seems I have to share the code between both machines, which means replicating the workers logic code in the web app code.

So:

• Is there a best practice to do that? Since code is redundant, I am thinking about using a git submodule (=> replicated in the web app code repo, and in the workers code repo)
• Should I better use something else than Celery then?
• Am I missing something?

Answer 1

You can use send_task. It takes same parameters than apply_async but you only have to give the task name. Without loading the module in django you can send tasks:

app.send_task('tasks.add', args=[2, 2], kwargs={})

http://celery.readthedocs.org/en/latest/reference/celery.html#celery.Celery.send_task

Answer 2

One way to manage this is to store your workers in your django project. Django and celery play nice to each other allowing you to use parts of your django project in your celery app. http://celery.readthedocs.org/en/latest/django/first-steps-with-django.html

Deploying this would mean that your web application would not use the modules involved with your celery workers, and on your celery machine your django views and such would never be used. This usually only results in a couple of megs of unused django application code...