CODEWITHSUNDEEP
pythonyamlpyyaml
Harley Holcombe
Harley Holcombe

Reputation: 181730

Save/dump a YAML file with comments in PyYAML

I have a yaml file that looks like this:

# The following key opens a door
key: value

Is there a way I can load and dump this data while maintaining the comment?

Upvotes: 84

Views: 66592

Answers (5)

Barney Szabolcs
Barney Szabolcs

Reputation: 12514

If you want to support multiple documents yaml,

import re

import ruamel.yaml  # pip install ruamel.yaml


def load_all(f):
    """
    Note: 
     - it does not handle the case when the whole document is just comment 
     - the dump is not as nice when the '---' is followed by a comment.
    """
    text = f.read()
    documents = re.split(r'\n---', text)
    yaml = ruamel.yaml.YAML()
    return [
        yaml.load(document)
        for document in documents
    ]


def dump_all(documents: list, f):
    yaml = ruamel.yaml.YAML()
    yaml.dump_all(documents, f)

Upvotes: 1

phihag
phihag

Reputation: 287755

PyYAML throws away comments at a very low level (in Scanner.scan_to_next_token).

While you could adapt or extend it to handle comments in its whole stack, this would be a major modification. Dumping (=emitting) comments seems to be easier and was discussed in ticket 114 on the old PyYAML bug tracker.

As of 2023, the feature request about adding support for loading comments is still stalling.

Upvotes: 36

Danail Kozhuharov
Danail Kozhuharov

Reputation: 31

If you are not constrained by a file schema, you can pick a specific key pattern to mean "ignored entry". For example - your yaml-data ingestion logic can filter out any entry with a key starting with '~' :

company:
  ~name: Must be the legal name
  name: Curious Adventures
  ~address: the official correspondence address
  address: 1234, New York, PO 1234

I have used this approach for JSON files as there we have the same issue with comments

Upvotes: 1

Anthon
Anthon

Reputation: 76568

If you are using block structured YAML, you can use the python package¹ ruamel.yaml which is a derivative of PyYAML and supports round trip preservation of comments:

import sys
import ruamel.yaml

yaml_str = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

yaml = ruamel.yaml.YAML()  # defaults to round-trip if no parameters given
code = yaml.load(yaml_str)
code['name']['given'] = 'Bob'

yaml.dump(code, sys.stdout)

with result:

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Note that the end-of-line comments are still aligned.

Instead of normal list and dict objects the code consists of wrapped versions² on which the comments attached.

¹ Install with pip install ruamel.yaml. Works on Python 2.6/2.7/3.3+
² ordereddict is used in case of a mapping, to preserve ordering

Upvotes: 133

Paul Ferrell
Paul Ferrell

Reputation: 51

I have a branch of pyyaml that does exactly this. https://github.com/pflarr/pyyaml

To build a yaml file with comments, you have to create an event stream that includes comment events. Comments are currently only allowed before sequence items and mapping keys.

This only currently works for python3, I haven't ported it to the python2 version of the library, but could easily do so on request. Additionally, this should also be fairly easy to port to the C libyaml as well, as the python code is a simple port of that anyway.

Upvotes: 5

Related Questions