CODEWITHSUNDEEP
pythonyamlpyyaml
KSS
KSS

Reputation: 167

How to load multiple yaml files with PyYAML?

I have a list of yaml files I want to load and return their contents in one single dictionary.

The easiest way would be to iterate through the filenames and load individually with safe_load, and then merge the resulting dictionaries. However,I was wondering if the python yaml library has already a functionality for that. I saw there are methods load_all and safe_load_all, but they still take only one stream, so I'm not sure if or how I can use those.

Upvotes: 4

Views: 10152

Answers (1)

holypriest
holypriest

Reputation: 321

This stream term is somewhat misleading. For the PyYaml library, you can think of it as a string (or a file) containing multiple yaml documents, such as this:

field1: foo
field2: bar
---
field1: baz
field2: qux
field3: fred
---
field4: fred

To use the load_all (or safe_load_all) for multiple documents split in multiple files, you will need to concatenate them, in some form. The function load_all itself is not able to read multiple files in a folder at once, right out of the box.

Long story short, you have two options:

  1. Use a loop or list comprehension to iterate over the files, calling the method read(), then using yaml.load over them individually

  2. Use a loop or list comprehension to iterate over the files and concatenate them in just one string with the format showed above (with documents separated by \n---\n), to process it using yaml.load_all

If you have a way to start with one single file and multiple documents inside (instead of multiple files), you can use yaml.load_all on that file directly.

It is important to notice that yaml.load_all returns a generator to you. Iterating over this generator brings you a dictionary for each document between ---. You can achieve your merging by assigning the first dictionary to a variable and calling the .update() method for each of the next dictionaries inside the generator:

gen = yaml.load_all(string)
starting_dict = next(gen)
for d in gen:
    starting_dict.update(d)

Upvotes: 3

Related Questions