CODEWITHSUNDEEP
pythonfb-hydraomegaconf
user17033672
user17033672

Reputation: 157

Calling a hydra app from a module rather than from the CLI

Usually to run a hydra app I would compose my app as follows:

my_package
|
├── config
│   ├── config.yaml
└── main.py

My main.py file might look like:

import hydra    
@hydra.main(version_base=None, config_path="conf", config_name="config")
def main(cfg: DictConfig):
    print('hello!')
    
if __name__ == '__main__':
    main()

to run it I simple run

python main.py in the CL in the relevant directory

This works great!

I have now been asked to productionise my code and port it to a docker container. Which comes with additional constraints I won't go into here, as such I have had to rebuild my code as a package. Now trying to run my app:

import my_package
import os
from importlib.resources import files
from omegaconf import OmegaConf

# load in my config from the non-code files included in the setup.py manifest
# as they are not allowed to exist on the docker container
cfg = OmegaConf.load(os.path.join(files("my_package"), 'conf', 'config.yaml'))

# start hydra app
my_package.main(cfg)

This does not work, and I am greeted with an UnsupportedInterpolationType: Unsupported interpolation type hydra error as I guess the config I supplied was not suitable.

I am aware of the compose API, which allows users to create their own configs independent of the @hydra.main process, however I am not trying to compose an config, I already have one, I simply want to be able to run the app without calling it from the CL.

Is there anyway to do this?

Happy to clarify anything that isn't clear

Helpful related questions:

Referring to Hydra's conf directory from a Python sub/sub-sub-directory module

How to load Hydra parameters from previous jobs (without having to use argparse and the compose API)?

Upvotes: 1

Views: 499

Answers (1)

Kathan Shah
Kathan Shah

Reputation: 1

Chances are that you are using config groups in your config.yaml, (although the dir structure you mention does not hint config groups). OmegaCfg does not compose the hierarchical hydra config. Instead use Hydra's Compose API

Update your wrapper code as follows:

from hydra import compose, initialize

with initialize(version_base=None, config_path=os.path.join(files("my_package"), 'conf')): # path to config.yaml
  cfg = compose(config_name="config") # config.yaml
  my_package.main(cfg)

Make sure you are using from hydra import compose, initialize and not using from hydra.experimental import compose, initialize.

Here is an example from hydra doc: Example

Upvotes: 0

Related Questions