Inline Comments ruamel.yaml

Question

I am using ruamel.yaml to add comments to my auto-config generation functionality for my personal project. Coming to the question, I am going to do this with a similar example to make it explicit.

---

Say, you have a default config dictionary (auto-generated):

{'auth': {'api_key': '', 'api_secret': ''}}

Now I load this dict into a CommentedMap object in the following way:

yaml_config = yaml.round_trip_load(yaml.round_trip_dump(dump))

Now, I have a metadata dict that includes multiline/single-line descriptions of these config values in the dict:

{
  "auth": {
    "api_key": "Think of this as the user name that represents your account when using the app.\n# Head over to XXX.com, register an app and get your auth keys!",
    "api_secret": "Think of this as the password that represents your account when using the app.\n# Head over to XXX.com, register an app and get your auth keys!"
  }
}

I now want to load over these metadata items and add comments to the respective fields in the yaml_config. Remember, the metadata and the dump schema are the same. The dump and metadata can be any amount of nested dicts, I only included an example above.

I tried the following code to make this work, but it just didn't work... No comments at all or any errors/exceptions.

previous_config[field_name].yaml_add_eol_comment(description, _field_name)

---

The correct output for the above example should be:

auth:
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_key: ''
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_secret: ''

Answer 1

I recommend to always include a full working example of what fails to do what you want, as now it is not completely clear exactly where things go wrong. New development using ruamel.yaml should not use the old API (round_trip_load/round_trip_dump), but use the new API (instantiating YAML()) That API was introduced several years ago and you give no indication that you are working with an ancient ruamel.yaml version that requires the use of the limited old API.

---

What you are trying to do can be trivially checked to be possible, by round tripping the end result YAML document and see that can be preserved unchanged:

import sys
import ruamel.yaml

yaml_str = """\
auth:
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_key: ''
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_secret: ''
"""

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4)
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)

which gives:

auth:
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_key: ''
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_secret: ''

You can inspect the comment attribute:

print(data['auth'].ca)

which prints:

Comment(comment=[None, [CommentToken('# Think of this as the password that represents your account when using the app.\n', line: 1, col: 4), CommentToken('# Head over to XXX.com, register an app and get your auth keys!\n', line: 2, col: 4)]],
  items={'api_key': [None, None, CommentToken('\n    # Think of this as the password that represents your account when using the app.\n    # Head over to XXX.com, register an app and get your auth keys!\n', line: 4, col: 4), None]})

End of line comments occur after other data, that is what the name implies. In ruamel.yaml end-of-line comments are usually associated with the key of a key-value pair and those are are expected to be on one and the same line, with each other, and the comment may extend to the following lines, including any empty lines. For ruamel.yaml and end-of-line comment does not necessarily have a comment at the end-of-the starting line (i.e. that can be empty).

You should also be aware that these (internal) routines are not fixed, will change and may stop to work, hopefully not, but sometimes without warning (so fix the version number in your install).

You are making things unnecessary difficult for yourself by associating the comment meta-data, that eventually has to come after the key api-key of key-value pair api_key: '', with key api-secret in your meta-data. I'll leave that self-induced problem unsolved.

yaml_add_eol_comment() always adds a missing initial #, cannot add a multi-line comment of which the "real" end-of-line part is emtpy, and requires you to indent the lines of a multi-line comment yourself.

You can also see in the output above, that the comment with regards to "user name" is attached completely different from the other comment, as this comment appears between the key (auth) and the value associated with that key (the associated value that is a two key value). Because of that, that comment needs to be added differently from the comment on key api_key:

import sys
import ruamel.yaml

yaml_str = """\
auth:
    api_key: ''
    api_secret: ''
"""

def indent_comment(s, indent):
    return s.replace('\n', '\n' + ' ' * indent)

MAPIND = 4

yaml = ruamel.yaml.YAML()
yaml.indent(mapping=MAPIND)
data = yaml.load(yaml_str)
auth = data['auth']
auth.yaml_set_start_comment(
    "Think of this as the user name that represents your account when using the app.\n"
    "# Head over to XXX.com, register an app and get your auth keys!",
    indent=MAPIND,
)
auth.yaml_add_eol_comment('place_holder', 'api_key')
# correct the assigned comment token
cai = auth.ca.items['api_key'][2]
cai.value = indent_comment('\n# Think of this as the password that represents your account when using the app.\n# Head over to XXX.com, register an app and get your auth keys!', MAPIND)

yaml.dump(data, sys.stdout)

which gives your required result:

auth:
    # Think of this as the user name that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_key: ''
    # Think of this as the password that represents your account when using the app.
    # Head over to XXX.com, register an app and get your auth keys!
    api_secret: ''

The above was done with ruamel.yaml==0.17.17 other version may need adapting.