CODEWITHSUNDEEP
pythonpydanticfb-hydraomegaconf
Kıvanç Yüksel
Kıvanç Yüksel

Reputation: 901

Is it possible to use Pydantic instead of dataclasses in Structured Configs in hydra-core python package?

Recently I have started to use hydra to manage the configs in my application. I use Structured Configs to create schema for .yaml config files. Structured Configs in Hyda uses dataclasses for type checking. However, I also want to use some kind of validators for some of the parameter I specify in my Structured Configs (something like this).

Do you know if it is somehow possible to use Pydantic for this purpose? When I try to use Pydantic, OmegaConf complains about it:

omegaconf.errors.ValidationError: Input class 'SomeClass' is not a structured config. did you forget to decorate it as a dataclass?

Upvotes: 12

Views: 6217

Answers (3)

user13199757
user13199757

Reputation:

I wanted to provide an update as Pydantic and Hydra have changed since this question was first posed.

Here is how to do this in 2024 using an updated (but similar) config.yaml file as above:

defaults:
  - config_schema
  - _self_

some_var: -1  # this will raise a ValueError

and a similar schema:

import hydra
from hydra.core.config_store import ConfigStore
from omegaconf import DictConfig, OmegaConf
from pydantic import field_validator
from pydantic.dataclasses import dataclass


@dataclass
class MyConfigSchema:
    some_var: float

    @field_validator("some_var")
    @classmethod
    def validate_data_dir(cls, v: float) -> float:
        if v < 0:
            raise ValueError(f"'some_var' can't be less than 0, got: {v}")
        return v


cs = ConfigStore.instance()
cs.store(name="config_schema", node=MyConfigSchema)


@hydra.main(
    version_base="1.3",
    config_path="conf/",
    config_name="config",
)
def main(cfg: DictConfig) -> None:
    """
    Main function for running experiments.

    :param cfg: Configuration object.
    :type cfg: DictConfig
    :return: None
    """
    validated_config = MyConfigSchema(**OmegaConf.to_container(cfg, resolve=True))
    print(validated_config)


if __name__ == "__main__":
    main()

Upvotes: 3

Kıvan&#231; Y&#252;ksel
Kıvan&#231; Y&#252;ksel

Reputation: 901

For those of you wondering how this works exactly, here is an example of it:

import hydra
from hydra.core.config_store import ConfigStore
from omegaconf import OmegaConf
from pydantic.dataclasses import dataclass
from pydantic import validator


@dataclass
class MyConfigSchema:
    some_var: float

    @validator("some_var")
    def validate_some_var(cls, some_var: float) -> float:
        if some_var < 0:
            raise ValueError(f"'some_var' can't be less than 0, got: {some_var}")
        return some_var


cs = ConfigStore.instance()
cs.store(name="config_schema", node=MyConfigSchema)


@hydra.main(config_path="/path/to/configs", config_name="config")
def my_app(config: MyConfigSchema) -> None:
    # The 'validator' methods will be called when you run the line below
    OmegaConf.to_object(config)


if __name__ == "__main__":    
    my_app()

And config.yaml :

defaults:
  - config_schema

some_var: -1  # this will raise a ValueError

Upvotes: 12

Jasha
Jasha

Reputation: 7799

See pydantic.dataclasses.dataclass, which are a drop-in replacement for the standard-library dataclasses with some extra type-checking.

Upvotes: 0

Related Questions