Ansible: Updating values in a list of dictionaries

Question

In my payload, I have a variable that is actually a list of dictionaries, such as this one:

myvar:
  - name: name1
    ip_addresses:
      - 10.10.10.10
      - 11.11.11.11
    nat_destination_addresses:
      - host: 12.12.12.12
        destination: 13.13.13.13
      - host: 14.14.14.14
        destination: 15.15.15.15
    nat_source_address: 16.16.16.16
    applications:
      - protocol: tcp
        port: 8302
      - protocol: udp
        port: 2000
      - protocol: tcp
        port: 2000-5600

  - name: name2
    ip_addresses:
      - 17.17.17.17

  - name: name3
    ip_addresses:
      - 18.18.18.18
      - 19.19.19.19

All the values for each element in myvar are optional, except for the name, which is mandatory.

I am trying to pad the ip addresses (ip_addresses, nat_destination_addresses and nat_source_address) and ports. The ports should have a length of five characters with zeroes at the beginning (2000 becomes 02000 and 2000-5600 becomes 02000-05600) and the ip addresses should have three characters for each subsection (18.18.18.18 becomes 018.018.018.018).

The problem that I have is that I am not able to change only subsections of myvar.

I have read other questions here, such as:

merging dictionaries in ansible

Using set_facts and with_items together in Ansible

But to no avail. No matter what I do, I am not able to keep the original dictionary, I end up with a list of ip_addresses if I use the combine filter from the second StackOverflow link.

The expected result is the original myvar variable with updated ip addresses and ports.

Answer 1

Larsks' answer was on point and is probably the best solution for most people, but my requirements are to limit the number of modules created with Python for this project, so here is my workaround for reference purposes.

Basically, what I do in this sample is:

Locally:

I take myvar, I output it to a yml file (with '---' at the top of the file and making sure that myvar is still set as the key.

Using regexp and the replace module, I replace the parts of the file that I want to replace.

On all of my hosts:

I reload the (now) properly formatted myvar and replace the old myvar variable using include_vars

---

- name: Customer {{ customer_id }} - Format the ip addresses and ports
  hosts: localhost
  gather_facts: no
  connection: local
  tags: [format_vars]

  tasks:
    - name: Copy the 'myvar' content to a local file to allow ip addresses 
and ports formatting
      copy:
        content: "---\n{{ { 'myvar': myvar} | to_nice_yaml(indent=2) }}"
        dest: "{{ formatted_myvar_file }}"

    - name: Pad all ip addresses parts with two zeroes to ensure that all parts have at least three numbers
      replace:
        path: "{{ formatted_myvar_file }}"
        regexp: '(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})'
        replace: '00\1.00\2.00\3.00\4'

    - name: Remove extra zeroes from ip addresses to ensure that all of their parts have exactly three numbers
      replace:
        path: "{{ formatted_myvar_file }}"
        regexp: '\d{0,2}(\d{3})\.\d{0,2}(\d{3})\.\d{0,2}(\d{3})\.\d{0,2}(\d{3})'
        replace: '\1.\2.\3.\4'

    - name: Pad all ports with four zeroes to ensure that they all have at least five numbers
      replace:
        path: "{{ formatted_myvar_file }}"
        regexp: 'port: (\d{1,5})'
        replace: 'port: 0000\1'

    - name: Remove extra zeroes from ports to ensure that they all have exactly five numbers
      replace:
        path: "{{ formatted_myvar_file }}"
        regexp: 'port: \d{0,4}(\d{5})'
        replace: 'port: \1'

    - name: Pad all second parts of port ranges with four zeroes to ensure that they all have at least five numbers
      replace:
        path: "{{ formatted_myvar_file }}"
        regexp: 'port: (\d{5})-(\d{1,5})'
        replace: 'port: \1-0000\2'

    - name: Remove extra zeroes from second parts of port ranges to ensure that they all have exactly five numbers
      replace:
        path: "{{ formatted_myvar_file }}"
        regexp: 'port: (\d{5})-\d{0,4}(\d{5})'
        replace: 'port: \1-\2'

- name: Customer {{ customer_id }} - Load the properly formatted ip addresses and ports
  hosts: localhost:all-n7k:srx-clu:all-mx80:all-vsrx
  gather_facts: no
  connection: local
  tags: [format_vars]

  tasks:
    - include_vars:
        file: "{{ formatted_myvar_file }}"
        ignore_errors: yes

Answer 2

This seems like a good time to throw your logic into a custom Ansible module. It doesn't have to be anything fancy, for example:

from ansible.module_utils.basic import AnsibleModule


def pad_addr(addr):
    return '.'.join('%03d' % int(x) for x in addr.split('.'))


def main():
    module_args = dict(
        data=dict(type='list', required=True),
    )

    module = AnsibleModule(
        argument_spec=module_args,
        supports_check_mode=True
    )

    data = module.params['data']

    for d in data:
        if 'ip_addresses' in d:
            d['ip_addresses'] = [pad_addr(x) for x in d['ip_addresses']]

        if 'nat_destination_addresses' in d:
            for dest in d['nat_destination_addresses']:
                dest['host'] = pad_addr(dest['host'])
                dest['destination'] = pad_addr(dest['destination'])

        if 'nat_source_address' in d:
            d['nat_source_address'] = pad_addr(d['nat_source_address'])

        if 'applications' in d:
            for service in d['applications']:
                service['port'] = '%05d' % service['port']

    module.exit_json(changed=False,
                     result=data)

if __name__ == '__main__':
    main()

If I drop the above into library/pad_data.py and then run the following playbook:

- hosts: localhost
  gather_facts: false

  vars:
    myvar:
      - name: name1
        ip_addresses:
          - 10.10.10.10
          - 11.11.11.11
        nat_destination_addresses:
          - host: 12.12.12.12
            destination: 13.13.13.13
          - host: 14.14.14.14
            destination: 15.15.15.15
        nat_source_address: 16.16.16.16
        applications:
          - protocol: tcp
            port: 8302
          - protocol: udp
            port: 2000
          - protocol: tcp
            port: 2000

      - name: name2
        ip_addresses:
          - 17.17.17.17

      - name: name3
        ip_addresses:
          - 18.18.18.18
          - 19.19.19.19

  tasks:

    - pad_data:
        data: "{{ myvar }}"
      register: padded

    - debug:
        var: padded.result

I get as the result:

TASK [debug] *******************************************************************
ok: [localhost] => {
    "padded.result": [
        {
            "applications": [
                {
                    "port": "08302", 
                    "protocol": "tcp"
                }, 
                {
                    "port": "02000", 
                    "protocol": "udp"
                }, 
                {
                    "port": "02000", 
                    "protocol": "tcp"
                }
            ], 
            "ip_addresses": [
                "010.010.010.010", 
                "011.011.011.011"
            ], 
            "name": "name1", 
            "nat_destination_addresses": [
                {
                    "destination": "013.013.013.013", 
                    "host": "012.012.012.012"
                }, 
                {
                    "destination": "015.015.015.015", 
                    "host": "014.014.014.014"
                }
            ], 
            "nat_source_address": "016.016.016.016"
        }, 
        {
            "ip_addresses": [
                "017.017.017.017"
            ], 
            "name": "name2"
        }, 
        {
            "ip_addresses": [
                "018.018.018.018", 
                "019.019.019.019"
            ], 
            "name": "name3"
        }
    ]
}