CODEWITHSUNDEEP
pythonpyyaml
filippo
filippo

Reputation: 5294

pyyaml map dict to dict of objects

I'm struggling with PyYAML docs to understand a probably easy thing. I have a dictionary that maps string names to python objects:

lut = { 'bar_one': my_bar_one_obj,
        'bar_two': my_bar_two_obj }

and I'd like to load a YAML file like this and map all "foo" nodes to my dictionary objects (the inverse, dumping, is not really necessary)

node1:
  # ...
  foo: "bar_one"
node2:
  # ...
  foo: "bar_two"

My first thought was to use add_constructor but I couldn't find a way to give it an extra kwarg. Maybe a custom loader? PyYAML docs aren't really helpful or probably I'm looking for the wrong keywords...

I could accept using a custom tag like

node1:
  # ...
  foo: !footag "bar_one"
node2:
  # ...
  foo: !footag "bar_two"

But detecting just foo nodes would be nicer

Upvotes: 0

Views: 1361

Answers (1)

Anthon
Anthon

Reputation: 76862

You are not looking for the wrong keywords, this is not something any of the YAML parsers I know of was made to do. YAML parsers load a, possible complex, data structure that is self contained. And what you want to do is merge that self contained structure, during one of the parsing steps, into an already existing structure ( lut ). The parser is built to allow tweaking by providing alternative routines not by providing routines + data

There is no option for that built into PyYAML, i.e. there is no built-in way to tell the loader about lut that make PyYAML do something with it, and certainly not to attach key-value pairs (assuming that is what you mean with the nodes) as values to its keys.

Probably the easiest way of getting what you want is using some post process which takes lut and the data loaded from your YAML file (which is also a dict) and combine the two.

If you want to try and do this with add_constructor, then what you need to do is construct a class with a __call__ method, create an instance of the class with lut as argument and than pass that instance in as an alternative constructor):

class ConstructorWithLut:
    def __init__(self, lut):
        self._lut = lut

    def __call__(self):
        # the actual constructor routine added by add_constructor

constructor_with_lut(lut)
SomeConstructor.add_constructor('your_tag', constructor_with_lut)

In which you can replace 'your_tag' with u'tag:yaml.org,2002:map' if you want your constructor to handle (all) normal dicts.

Another option is to do this during YAML loading, but once again you cannot just tweak the Loader, or one of its constituent components (the Constructor) as you normally hand in the class not an object. You need an object to be able to attach lut. So what you would to do is create your own constructor and your own loader that uses that constructor and then a load() replacement that instantiates your loader, attaches lut (by just adding it as a unique attribute, or by passing it in as a parameter and handing it on to your constructor).

Your constructor, which should be a subclass of one of the existing constructors, then has to have its own construct_mapping() that first calls the parent class' construct_mapping() and, before returning the result, inspects whether it could update that attribute to which lut has been assigned. You cannot do this based on looking at the keys of the dict for foo, because if you have such a key you don't have access to the parent node that you need to assign to lut. What you need to do is see if any of the values of the mapping is a dict that has a key name foo, and if it does the dictionary can be used to update lut based on the value associated with foo.

I would certainly first implement the post process stage using two routines:

def update_from_yaml(your_dict, yaml_data):
    for node_key in yaml_data:
        node_value = yaml_data[node_key]
        map_value(your_dict, node_key, node_value)

def map_value(your_dict, key, value):
    foo_val = value.get('foo')
    if foo_val is None:  # key foo not found
        return
    your_dict[foo_val] = value  #  or  = {key: value}

I am not sure what you really mean with "assigning all foo nodes", the YAML data has no nodes at the top level, it only has keys and values. So you either assign that pair or only its value (a dict).

Once those two routines work satisfactory, you can try to implement the add_constructor or Loader based alternatives, in which you should be able to re-use at least map_value

Upvotes: 2

Related Questions