Ansible: convert content of list into list of dicts with data manipulation

Question

community, I'm trying to convert the list to the desired data structure, so I could work with it later:

output of the previous task:

ok: [ubuntu_vm] => {
    "msg": [
        "[[machine]]",
        "  name = \"foo\"",
        "  year = \"bar\"",
        "[[machine]]",
        "  name = \"notfoo\"",
        "  year = \"notbar\"",
        "[[machine]]",
        "  name = \"smth\"",
        "  year = \"else\"",
    ]
}

How can I achieve the desired structure as list of dictionaries, so that I could iterate through it using selectattr() filter?

[
 - {"name": "foo", "year": "bar"}
 - {"name": "notfoo", "year": "notbar" }
 - ...
]

I've started with smth like this, but it produces lots of garbage as well:

{{ name_exists.stdout_lines | string | split('[[machine]]') | list }}

And after that I believe that I should apply smth like:

{{ existing_runners + [ { item.split(',')[1]: item.split(',')[2] } ] }}

Answer 1

Given the data is a list of strings

  data:
  - '[[machine]]'
  - '  name = "foo"'
  - '  year = "bar"'
  - '[[machine]]'
  - '  name = "notfoo"'
  - '  year = "notbar"'
  - '[[machine]]'
  - '  name = "smth"'
  - '  year = "else"'

Create lists of names and years

  names: "{{ data|select('match', '^.*name\\s*=.*$')|
                  map('split', '=')|map('last')|map('trim')|
                  list }}"
  years: "{{ data|select('match', '^.*year\\s*=.*$')|
                  map('split', '=')|map('last')|map('trim')|
                  list }}"

gives

  names: ['"foo"', '"notfoo"', '"smth"']
  years: ['"bar"', '"notbar"', '"else"']

Then strip the double-quotes and create a YAML dictionary

  regex: '^"*(.*?)"*$'
  replace: '\1'
  names_strip: "{{ names|map('regex_replace', regex, replace)|list }}"
  years_strip: "{{ years|map('regex_replace', regex, replace)|list }}"
  content_dict: "{{ dict(names_strip|zip(years_strip)) }}"

gives

  content_dict:
    foo: bar
    notfoo: notbar
    smth: else

Next, you can convert it to a list of dictionaries

  content: "{{ content_dict|dict2items(key_name='name',
                                       value_name='year') }}"

gives the desired structure

  content:
    - {name: foo, year: bar}
    - {name: notfoo, year: notbar}
    - {name: smth, year: else}

Create the list without the intermediary dictionary if the names are not unique. The declaration below will create the desired structure too

  content: "{{ names_strip|zip(years_strip)|
                           map('zip', ['name', 'year'])|
                           map('map', 'reverse')|
                           map('community.general.dict')|
                           list }}"

Q: "How can I transform it, so that it can be used content | selectattr(...)?" A: No transformation is needed. For example,

  content_foo: "{{ content|selectattr('name', 'search', 'foo') }}"

gives as expected

  content_foo:
    - {name: foo, year: bar}
    - {name: notfoo, year: notbar}

---

Notes

• Example of a complete playbook

- hosts: localhost

  vars:
    data: [
        "[[machine]]",
        "  name = \"foo\"",
        "  year = \"bar\"",
        "[[machine]]",
        "  name = \"notfoo\"",
        "  year = \"notbar\"",
        "[[machine]]",
        "  name = \"smth\"",
        "  year = \"else\"",
        ]
    names: "{{ data|select('match', '^.*name\\s*=.*$')|
                    map('split', '=')|map('last')|map('trim')|
                    list }}"
    years: "{{ data|select('match', '^.*year\\s*=.*$')|
                    map('split', '=')|map('last')|map('trim')|
                    list }}"
    regex: '^"*(.*?)"*$'
    replace: '\1'
    names_strip: "{{ names|map('regex_replace', regex, replace)|list }}"
    years_strip: "{{ years|map('regex_replace', regex, replace)|list }}"
    content_dict: "{{ dict(names_strip|zip(years_strip)) }}"
    content: "{{ content_dict|dict2items(key_name='name',
                                         value_name='year') }}"

  tasks:
    - debug:
        var: content

---

• The code will be much simpler if the blocks are unique. For example,

  data:
  - '[[machine1]]'
  - '  name = "foo"'
  - '  year = "bar"'
  - '[[machine2]]'
  - '  name = "notfoo"'
  - '  year = "notbar"'
  - '[[machine3]]'
  - '  name = "smth"'
  - '  year = "else"'

It will be possible to parse the INI data

  machines: "{{ data|join('\n')|community.general.jc('ini') }}"

gives

  machines:
    '[machine1': {name: foo, year: bar}
    '[machine2': {name: notfoo, year: notbar}
    '[machine3': {name: smth, year: else}

and create the desired structure

  content: "{{ machines.values()|list }}"