CODEWITHSUNDEEP
pythonyaml
junhuizh
junhuizh

Reputation: 412

How to use a key's value for another key's value in yaml?

I am working on reading parameters from a yaml file, and I want to use a value to produce another value. For example, I need the following parameters a and b, and b uses the value of a.

A:
    a: 10
    b: a * 10
B:
    ......

And here is how I process the file:

from ruamel.yaml import safe_load, YAMLError

with open(yaml_file, 'r', encoding="utf-8") as hp:
    try:
        yaml_hparams = safe_load(hp)
        for _, V in yaml_hparams.items():
            for k, v in V.items():
                hparams[k] = v
    except YAMLError:
        print(YAMLError)

Obviously this gives a string of "a * 10" to b's value. How can I give a * 10, i.e. 100, to b?

Upvotes: 2

Views: 5418

Answers (2)

martineau
martineau

Reputation: 123463

Here's a simple proof of concept to show the feasibility of using the built-in eval() function to do it, as I suggested in one of my comments. As such, it has some limitations such as not being able to handle "forward references", but I think many of those could be overcome, if required.

import json
from ruamel.yaml import safe_load, YAMLError


yaml_file = 'test_refs.yaml'

hparams = {}
with open(yaml_file, 'r', encoding="utf-8") as hp:
    try:
        yaml_hparams = safe_load(hp)
        for _, V in yaml_hparams.items():
            for k, v in V.items():
                hparams[k] = v
    except YAMLError as exc:
        print(exc)

print('Before:')
print(json.dumps(hparams, indent=4))   # Pretty-print result.

# Interpolate values with eval()
hparams['__builtins__'] = None  # Prevents access to built-ins.

for key, value in list(hparams.items()):
    if isinstance(value, str):
        try:
            hparams[key] = eval(value, hparams)
        except (NameError, TypeError):
            pass  # Possible forward-reference, ignore.

del hparams['__builtins__']  # No longer needed.

print()
print('After:')
print(json.dumps(hparams, indent=4))

Upvotes: 0

flyx
flyx

Reputation: 39688

Not with YAML. YAML is a language for data serialization, it is not a programming language.

You have multiple options. One is to use a templating language like Jinja to preprocess your YAML and calculate the value. Preprocessing YAML with Jinja is done in tools like Ansible and SaltStack, so it is a rather common practice. However, be aware that Jinja does not understand the structure of YAML and you need to be careful about whitespace.

Example:

{% set val = 10 %}
A:
    a: {{ val }}
    b: {{ val * 10 }}

Obviously, you will need to call Jinja on the input before you load the YAML then.

Another option is to use YAML tags to tell your code how to post-process the values, and implement it there:

A:
    a: &val 10
    b: !prod [*val, 10]

Code would be something like

from ruamel.yaml import safe_load, SafeLoader, YAMLError
import functools

def prod_constructor(loader, node):
    return functools.reduce(lambda x, y: x * y, loader.construct_sequence(node))

SafeLoader.add_constructor(u'!prod', prod_constructor)

with open(yaml_file, 'r', encoding="utf-8") as hp:
try:
    yaml_hparams = safe_load(hp)
    for _, V in yaml_hparams.items():
        for k, v in V.items():
            hparams[k] = v
except YAMLError:
    print(YAMLError)

Upvotes: 1

Related Questions