CODEWITHSUNDEEP
dictionaryansiblenestedjinja2
Joshua Gilman
Joshua Gilman

Reputation: 1202

Flatten nested dictionary to key/value pairs with Ansible

---
- hosts: localhost
  vars:
    mydict:
      key1: val1
      key2: val2
      key3:
        subkey1: subval1
        subkey2: subval2
  tasks:
    - debug:
        msg: "{{ TODO }}"

How would I make the above debug message print out all key/value pairs from the nested dictionary? Assume the depth is unknown. I would expect the output to be something like:

{
  "key1": "val1",
  "key2": "val2",
  "subkey1": "subval1"
  "subkey2": "subval2"
}

Upvotes: 3

Views: 1416

Answers (1)

Vladimir Botka
Vladimir Botka

Reputation: 68254

Write a filter plugin and use pandas.json_normalize, e.g.

shell> cat filter_plugins/dict_normalize.py 
from pandas.io.json import json_normalize

def dict_normalize(d):
    df = json_normalize(d)
    l = [df.columns.values.tolist()] + df.values.tolist()
    return(l)

class FilterModule(object):
    def filters(self):
        return {
            'dict_normalize': dict_normalize,
        }

The filter returns lists of keys and values

    - set_fact:
        mlist: "{{ mydict|dict_normalize }}"

gives

  mlist:
  - - key1
    - key2
    - key3.subkey1
    - key3.subkey2
  - - val1
    - val2
    - subval1
    - subval2

Create a dictionary, e.g.

    - debug:
        msg: "{{ dict(mlist.0|zip(mlist.1)) }}"

gives

  msg:
    key1: val1
    key2: val2
    key3.subkey1: subval1
    key3.subkey2: subval2

If the subkeys are unique remove the path

    - debug:
        msg: "{{ dict(_keys|zip(mlist.1)) }}"
      vars:
        _regex: '^(.*)\.(.*)$'
        _replace: '\2'
        _keys: "{{ mlist.0|map('regex_replace', _regex, _replace)|list }}"

gives

  msg:
    key1: val1
    key2: val2
    subkey1: subval1
    subkey2: subval2

Notes

  • Install the package, e.g. python3-pandas

  • The filter might be extended to support all parameters of json_normalize

  • The greedy regex works also in nested dictionaries

Q: "Turn the key3.subkey1 to get the original dictionary."

A: Use json_query. For example, given the dictionary created in the first step

    - set_fact:
        mydict_flat: "{{ dict(mlist.0|zip(mlist.1)) }}"

  mydict_flat:
    key1: val1
    key2: val2
    key3.subkey1: subval1
    key3.subkey2: subval2

iterate the keys and retrieve the values from mydict

    - debug:
        msg: "{{ mydict|json_query(item) }}"
      loop: "{{ mydict_flat|list }}"

gives

  msg: val1
  msg: val2
  msg: subval1
  msg: subval2

Upvotes: 3

Related Questions