CODEWITHSUNDEEP
pythonyamlconfigceleryloader
Taras Voinarovskyi
Taras Voinarovskyi

Reputation: 176

Configuring celery with a .conf file

Good day.

I set up a separate project from the main one called myproject-celery. It is a buildout based project which contains the async part of my project. For convenience I want to have a file, that will be containing this machine's configuration. I know that celery provides the python config file, but I do not like this configuration style.

Let's say I have a configuration in a Yaml config file named myproject.yaml

What I want to achieve: 

./bin/celery worker --config /absolute/path/to/project/myproject.yaml --app myproject.celery

The problem really is that I want to specify the file's location, because it can change. I tried writing a custom loader class, but I failed, cause I do not even know why and when the many custom methods of this class are called (the only doc that I found is http://docs.celeryproject.org/en/latest/reference/celery.loaders.base.html?highlight=loader#id1 and It's no help for me). I tried to do something on import phase for the app module, but I can not pass the filepath to that module's code... The only solution that I came up with was using a custom ENV param that will contain the path, but I do not see why can't it be a launch param like in most apps, that I use(refering to pyramid with it's paster serve myproject.ini)

So the question: What do I have to do to set up the config from a file that I could specify by an absolute path?

EDIT: The question was not answered, sow I posted an issue on celery's github. Will wait for a response. https://github.com/celery/celery/issues/1100

Upvotes: 2

Views: 1850

Answers (1)

Sean Vieira
Sean Vieira

Reputation: 159895

Looking at celery.loaders.base it looks like the method you want to override is read_configuration:

from celery.datastructures import DictAttr
from celery.loaders.base import BaseLoader


class YAMLLoader(BaseLoader):
    def read_configuration():
        # Load YAML file here and return a DictAttr instance

Upvotes: 1

Related Questions