CODEWITHSUNDEEP
yamlhydra
James Arten
James Arten

Reputation: 666

A question about formatting syntax in `yaml`

I'm trying to understand some yaml syntax related to using hydra in a machine learning approach. So given the following, extracted from the original github repo:

datamodule:
  _target_: cdvae.pl_data.datamodule.CrystDataModule

  datasets:
    train:
      _target_: cdvae.pl_data.dataset.CrystDataset
      name: Formation energy train
      path: ${data.root_path}/train.csv
      prop: ${data.prop}
      niggli: ${data.niggli}
      primitive: ${data.primitive}
      graph_method: ${data.graph_method}
      lattice_scale_method: ${data.lattice_scale_method}
      preprocess_workers: ${data.preprocess_workers}

I don't understand what the syntax ${} stands for in this context. Is that some kind of text formatting? Or is it calling some specific module? Could anybody provide some examples?

EDIT:

Ok, as pointed by @flyx, it seems that the syntax ${} refers to hydra which is indeed part of the project I'm analyzing.

Upvotes: 0

Views: 659

Answers (1)

Jasha
Jasha

Reputation: 7729

The string foo: ${bar} in yaml corresponds roughly to the dict {"foo": "${bar}"} in python. This is to say, the dollar-bracket notation ${} is not given special meaning by the yaml grammar.

OmegaConf, which is the backend used by Hydra, does give special meaning to the dollar-bracket syntax; it is used for "variable interpolation." See the OmegaConf docs on variable interpolation.

To summarize briefly, the idea of variable interpolation is that the string "${bar}" is a "pointer" to the value with key bar that would appear elsewhere in your OmegaConf config object.

Here's a short demo of variable interpolations in action:

from omegaconf import OmegaConf
yaml_data = """
foo: ${bar}
bar: baz
"""
config = OmegaConf.create(yaml_data)
assert config.bar == "baz"
assert config.foo == "baz"  # the ${bar} interpolation points to the "baz" value

Upvotes: 2

Related Questions