CODEWITHSUNDEEP
pythonpython-3.xfb-hydra
Alb
Alb

Reputation: 1232

FB-Hydra: calling compose during hydra main is not properly finding configs in the searchpath

I'm trying to call compose within hydra.main, but it is not properly using the searchpath I set in my main config. This is the organization of my script:

my_script
 ┣ config
 ┃ ┣ config.yaml
 ┗ my_script.py

My scripts depend on a config package within a library I'm developing, It is organized as follows:

my_lib
 ┣ config
 ┃ ┣ config_group_A
 ┃ ┃ ┣ ...
 ┃ ┣ config_group_B
 ┃ ┃ ┣ some_config.yaml
 | | ┗ __init__.py
 | ┗ __init__.py

and so on.

In my my_script/config/config.yaml I added the following:

  searchpath:
    - pkg://my_lib.config
    - pkg://my_lib.config.config_group_B

The purpose is that the main config uses a lot of configs within my_lib/config, but also I want to, within hydra.main, use hydra.compose to compose the config some_config.yaml

The problem is that when I call hydra.compose within hydra.main, I get the following error:

hydra.errors.MissingConfigException: Cannot find primary config 'some_config'. Check that it's in your config search path.

Config search path:
    provider=hydra, path=pkg://hydra.conf
    provider=main, path=file:///<path_to_scripts>/my_script/config
    provider=schema, path=structured://

But I'm confused because I added both my_lib.config and my_lib.config.config_group_B to the searchpath. These even appear in the hydra.core.hydra_config.HydraConfig.get().searchpath and hydra.core.hydra_config.HydraConfig.get().runtime.config_sources, so it is strange I cannot compose some_config.yaml during hydra.main. For refernece, I call in the script hydra.main(version_base="1.3", config_path='config', config_name='config')

What is the issue here?

Upvotes: 0

Views: 119

Answers (1)

Omry Yadan
Omry Yadan

Reputation: 33696

This is likely a mistake:

In my my_script/config/config.yaml I added the following:

searchpath: - pkg://my_lib.config - pkg://my_lib.config.config_group_B

Search path elements should not be nested. if you want to compose configs in config_group_B you should just refer to them with their full relative path in your search path, e.g config_group_B/some-config.

About your actual quesiton: Can you provide a complete minimal runnable example (code and configs)?

Upvotes: 0

Related Questions