CODEWITHSUNDEEP
pythonyaml
link89
link89

Reputation: 1793

Is it possible to remove unecessary nested structure in yaml file?

I need to set a param that is deep inside a yaml object like below:

executors:
  hpc01:
    context:
      cp2k:
        charge: 0

Is it possible to make it more clear, for example

executors: hpc01: context: cp2k: charge: 0

I am using ruamel.yaml in Python to parse the file and it fails to parse the example. Is there some yaml dialect can support such style, or is there better way to write such configuration in standard yaml spec?

Upvotes: 1

Views: 98

Answers (2)

Anthon
Anthon

Reputation: 76614

What you are proposing is invalid YAML, since the colon + space is parsed as a value indicator. Since YAML can have mappings as keys for other mappings, you would get all kinds of interpretation issues, such as should

a: b: c

be interpreted as a mapping with key a: b and value c or as a mapping with key a and value b: c.

If you want to write everything on one line, and don't want the overhead of YAML's flow-style, I suggest you use the fact that the value indicator expects a space after the colon and do a little post-processing:

import sys
import ruamel.yaml

yaml_str = """\
before: -1
executors:hpc01:context:cp2k:charge: 0
after: 1
"""

COLON = ':'

def unfold_keys(d):
    if isinstance(d, dict):
        replace = []
        for idx, (k, v) in enumerate(d.items()):
            if COLON in k:
                for segment in reversed(k.split(COLON)):
                    v = {segment: v}
                replace.append((idx, k, v))
            else:
                unfold_keys(v)
        for idx, oldkey, kv in replace:
            del d[oldkey]
            v = list(kv.values())[0]
            # v.refold = True
            d.insert(idx, list(kv.keys())[0], v)
    elif isinstance(d, list):
        for elem in d:
            unfold_keys
    return d
    
yaml = ruamel.yaml.YAML()
data = unfold_keys(yaml.load(yaml_str))
yaml.dump(data, sys.stdout)

which gives:

before: -1
executors:
  hpc01:
    context:
      cp2k:
        charge: 0
after: 1

Since ruamel.yaml parses mappings in the default mode to CommentedMap instances which have .insert() method, you can actually preserve the position of the "unfolded" key in the mapping.

You can of course use another character (e.g. underscore). You can also reverse the process by uncommenting the line # v.refold = True and provide another recursive function that walks over the data and checks on that attribute and does the reverse of unfold_keys(), just before dumping.

Upvotes: 1

Grady Player
Grady Player

Reputation: 14549

since all json is valid yaml...

executors: {"hpc01" : {"context": {"cp2k": {"charge": 0}}}}

should be valid...

a little proof:

from ruamel.yaml import YAML

a = YAML().load('executors: {"hpc01" : {"context": {"cp2k": {"charge": 0}}}}')

b = YAML().load('''executors:
  hpc01:
    context:
      cp2k:
        charge: 0''')
        
if a == b:
    print ("equal")

will print: equal.

Upvotes: 2

Related Questions