CODEWITHSUNDEEP
pythonyamlruamel.yaml
gschizas
gschizas

Reputation: 90

Is it possible to have aliases in a multi-document YAML stream that span all documents?

This is what I'm trying to do (code is in Python 3):

import ruamel.yaml as yaml
from print import pprint

yaml_document_with_aliases = """
title: test
choices: &C
  a: one
  b: two
  c: three
---
title: test 2
choices: *C
"""

items = list(yaml.load_all(yaml_document_with_aliases))

The result is:

ComposerError: found undefined alias 'C'

When I'm using a non-document-based YAML file, this works as expected:

import ruamel.yaml as yaml
from print import pprint

yaml_nodes_with_aliases = """
-
  title: test
  choices: &C
    a: one
    b: two
    c: three
-
  title: test 2
  choices: *C
"""

items = yaml.load(yaml_nodes_with_aliases)

pprint(items)

Result:

[{'choices': {'a': 'one', 'b': 'two', 'c': 'three'}, 'title': 'test'},
 {'choices': {'a': 'one', 'b': 'two', 'c': 'three'}, 'title': 'test 2'}]

(what I wanted to accomplish anyway)

Since it's not possible right now, I'm using the following brittle workaround:

def yaml_load_all_with_aliases(yaml_text):
    if not yaml_text.startswith('---'):
        yaml_text = '---\n' + yaml_text
    for pat, repl in [('^', '  '), ('^\s*---\s*$', '-'), ('^\s+\.{3}$\n', '')]:
        yaml_text = re.sub(pat, repl, yaml_text, flags=re.MULTILINE)
    yaml_text = yaml_text.strip()
    return yaml.safe_load(yaml_text)

Upvotes: 4

Views: 3164

Answers (1)

Anthon
Anthon

Reputation: 76599

The problem here is that:

title: test
choices: &C
  a: one
  b: two
  c: three
---
title: test 2
choices: *C

is not a document, these are two YAML documents in one file. The anchor definition &C doesn't carry from one YAML document to another, it can only be used up until the document separator ---.

If you are willing to have all anchors "carry over" to the following documents in a single YAML stream you can graft a new compose_document method on the Composer class (i.e. monkey-patch it):

import sys
import ruamel.yaml

yaml_str = """\
title: test
choices: &C
  a: one
  b: two
  c: three
---
title: test 2
choices: *C
"""


def my_compose_document(self):
    self.get_event()
    node = self.compose_node(None, None)
    self.get_event()
    # this prevents cleaning of anchors between documents in **one stream**
    # self.anchors = {}
    return node

ruamel.yaml.composer.Composer.compose_document = my_compose_document

datas = []
for data in ruamel.yaml.safe_load_all(yaml_str):
    datas.append(data)

datas[0]['choices']['a'] = 1
for data in datas:
    ruamel.yaml.round_trip_dump(data, sys.stdout, explicit_start=True)

which gives:

---
title: test
choices:
  a: 1
  b: two
  c: three
---
title: test 2
choices:
  a: one
  b: two
  c: three

Note that this gets you a copy of the dict with keys a, b, and c.

(If the key ordering, and preservation of comments is important, use round_trip_load_all instead of safe_load_all)

Upvotes: 3

Related Questions