CODEWITHSUNDEEP
pythongitlab-cipyyaml
conspicillatus
conspicillatus

Reputation: 335

Load .gitlab-ci.yml with pyyaml fails with could not determine constructor

Loading .gitlab-ci.yml fails when I try to load it with yaml.load.

yaml.constructor.ConstructorError: could not determine a constructor for the tag '!reference'
  in "python.yml", line 9, column 7

That's the yaml I try to load.

.python:
  before_script:
    - pip install -r requirements.txt
  script:
    - echo "Hello Python!"

test:
  before_script:
    - !reference [.python, before_script]
  script:
    - pytest

Right now I'm not interested in those references. However, I'm modifying some parts of the yaml and then write it back to the filesystem. So, I don't want to strip these references.

Upvotes: 3

Views: 1202

Answers (2)

conspicillatus
conspicillatus

Reputation: 335

In the meantime, I have figured out how to add a constructor and representer to load and dump YAML with custom tags.

import yaml

class Reference:
    yaml_tag = u'!reference'
    def __init__(self, values):
        self._values = values

def reference_constructor(loader: yaml.SafeLoader, node: yaml.nodes.CollectionNode) -> Reference:
    return Reference(loader.construct_sequence(node))

def reference_representer(dumper: yaml.SafeDumper, ref: Reference) -> yaml.nodes.SequenceNode:
    return dumper.represent_sequence(Reference.yaml_tag, ref._values)

def loader():
    loader = yaml.SafeLoader
    loader.add_constructor(Reference.yaml_tag, reference_constructor)
    return loader

def dumper():
    dumper = yaml.SafeDumper
    dumper.add_representer(Reference, reference_representer)
    return dumper

def rewrite(ci_file):
    with open(ci_file) as f:
        ci_yaml = yaml.load(f, Loader=loader())
    print(ci_yaml)
    with open(ci_file, "w") as f:
        yaml.dump(ci_yaml, f, Dumper=dumper())

if __name__ == "__main__":
    rewrite(".gitlab-ci.yml")

Upvotes: 1

Kiran S
Kiran S

Reputation: 178

Upgrade to ruamel.yaml>=0.15.x

    yaml_str = """
.python:
  before_script:
    - pip install -r requirements.txt
  script:
    - echo "Hello Python!"

test:
  before_script:
    - !reference [.python, before_script]
  script:
    - pytest """

from ruamel.yaml import YAML
yaml = YAML()
yaml.preserve_quotes = True
data = yaml.load(yaml_str)

Upvotes: 1

Related Questions