CODEWITHSUNDEEP
ansible
WarrenG
WarrenG

Reputation: 1850

Ansible split comma seperated values into json

Using ansible I am pulling a json object of specific security groups from AWS:

  amazon.aws.ec2_group_info:
    filters:
      "tag:vpn_ports": "*"
  register: sec_group_info_output

With this I am then creating my own json output that I will use later on using:

  set_fact:
    vpn_groups: "{{ vpn_groups|default({}) | combine( {item.group_id: item.tags.vpn_ports}) }}"
  with_items: "{{ sec_group_info_output | json_query('security_groups') }}"

The output of the above is:

"vpn_groups": {
        "sg-123456": "5432,22,53",
        "sg-987654": "22",
        "sg-18374923": "22,3389"
    }

What I am now trying to do is to have an output that looks like the below:

"vpn_groups": {
        "sg-123456": "5432",
        "sg-123456": "22",
        "sg-123456": "53",
        "sg-987654": "22",
        "sg-18374923": "22",
        "sg-18374923": "3389"
    }

The comma seperated values have been split across multiple items. I'm not sure how to go about doing this.

Edit: My end goal is to create new security group rules for each security group_id above with the respective ports found in item.tags.vpn_ports.

Perhaps a suitable alternative would be to have the list look something like:

{
    "group_id": "sg-123456",
    "port": "5432"
},
{
    "group_id": "sg-123456",
    "port": "22"
},
{
    "group_id": "sg-123456",
    "port": "53"
},
{
    "group_id": "sg-987654",
    "port": "22"
},
...etc

Upvotes: 0

Views: 528

Answers (2)

Vladimir Botka
Vladimir Botka

Reputation: 68074

Put the below declaration into the vars

  vpn_groups: |
    [
    {% for i in sec_group_info_output.security_groups %}
    {% for j in i.tags.vpn_ports.split(',') %}
    {"group_id": "{{ i.group_id }}", "port": "{{ j }}"},
    {% endfor %}
    {% endfor %}
    ]

gives what you want

  vpn_groups:
  - group_id: sg-123456
    port: '5432'
  - group_id: sg-123456
    port: '22'
  - group_id: sg-123456
    port: '53'
  - group_id: sg-987654
    port: '22'
  - group_id: sg-18374923
    port: '22'
  - group_id: sg-18374923
    port: '3389'

Example of a complete playbook for testing

- hosts: localhost

  vars:

    sec_group_info_output:
      security_groups:
        - group_id: sg-123456
          tags:
            vpn_ports: "5432,22,53"
        - group_id: sg-987654
          tags:
            vpn_ports: "22"
        - group_id: sg-18374923
          tags:
            vpn_ports: "22,3389"

    vpn_groups: |
      [
      {% for i in sec_group_info_output.security_groups %}
      {% for j in i.tags.vpn_ports.split(',') %}
      {"group_id": "{{ i.group_id }}", "port": "{{ j }}"},
      {% endfor %}
      {% endfor %}
      ]

  tasks:

    - debug:
        var: vpn_groups

The next option is creating a dictionary. For example, put the below declarations into the vars

  vpn_keys: "{{ sec_group_info_output.security_groups|
                map(attribute='group_id')|list }}"
  vpn_vals: "{{ sec_group_info_output.security_groups|
                map(attribute='tags.vpn_ports')|
                map('split', ',')|list }}"
  vpn_dict: "{{ dict(vpn_keys|zip(vpn_vals)) }}"

gives

  vpn_dict:
    sg-123456:
    - '5432'
    - '22'
    - '53'
    sg-18374923:
    - '22'
    - '3389'
    sg-987654:
    - '22'

This dictionary can be iterated

    - debug:
        msg: "{{ item.0.key }} {{ item.1 }}"
      with_subelements:
        - "{{ vpn_dict|dict2items }}"
        - value

gives (abridged)

  msg: sg-123456 5432
  msg: sg-123456 22
  msg: sg-123456 53
  msg: sg-987654 22
  msg: sg-18374923 22
  msg: sg-18374923 3389

Example of a complete playbook for testing

- hosts: localhost

  vars:

    sec_group_info_output:
      security_groups:
        - group_id: sg-123456
          tags:
            vpn_ports: "5432,22,53"
        - group_id: sg-987654
          tags:
            vpn_ports: "22"
        - group_id: sg-18374923
          tags:
            vpn_ports: "22,3389"

    vpn_keys: "{{ sec_group_info_output.security_groups|
                  map(attribute='group_id')|list }}"
    vpn_vals: "{{ sec_group_info_output.security_groups|
                  map(attribute='tags.vpn_ports')|
                  map('split', ',')|list }}"
    vpn_dict: "{{ dict(vpn_keys|zip(vpn_vals)) }}"

  tasks:

    - debug:
        var: vpn_dict

    - debug:
        msg: "{{ item.0.key }} {{ item.1 }}"
      with_subelements:
        - "{{ vpn_dict|dict2items }}"
        - value

Upvotes: 0

larsks
larsks

Reputation: 311740

This will work better if you structure your vpn_groups variable as a list of dictionaries, each with a ports key containing a list of ports. That will allow us to use the subelements filter to do what you want. E.g., something like this:

- hosts: localhost
  gather_facts: false
  vars:
    sec_group_info_output:
      security_groups:
        - group_id: sg-123456
          tags:
            vpn_ports: "5432,22,53"
        - group_id: sg-987654
          tags:
            vpn_ports: "22"
        - group_id: sg-18374923
          tags:
            vpn_ports: "22,3389"
  tasks:
    - set_fact:
        vpn_groups: "{{ vpn_groups + [{'group_id': item.group_id, 'ports': item.tags.vpn_ports.split(',')}] }}"
      vars:
        vpn_groups: []
      with_items: "{{ sec_group_info_output | json_query('security_groups') }}"


    - debug:
        msg: "open port {{ item.1 }} for group {{ item.0.group_id }}"
      loop: "{{ vpn_groups|subelements('ports') }}"

The set_fact task here builds a structure like this:

[
  {
    "group_id": "sg-123456",
    "ports": [
      "5432",
      "22",
      "53"
    ]
  },
  {
    "group_id": "sg-987654",
    "ports": [
      "22"
    ]
  },
  {
    "group_id": "sg-18374923",
    "ports": [
      "22",
      "3389"
    ]
  }
]

This will output:

PLAY [localhost] ***************************************************************

TASK [set_fact] ****************************************************************
ok: [localhost] => (item={'group_id': 'sg-123456', 'tags': {'vpn_ports': '5432,22,53'}})
ok: [localhost] => (item={'group_id': 'sg-987654', 'tags': {'vpn_ports': '22'}})
ok: [localhost] => (item={'group_id': 'sg-18374923', 'tags': {'vpn_ports': '22,3389'}})

TASK [debug] *******************************************************************
ok: [localhost] => (item=[{'group_id': 'sg-123456', 'ports': ['5432', '22', '53']}, '5432']) => {
    "msg": "open port 5432 for group sg-123456"
}
ok: [localhost] => (item=[{'group_id': 'sg-123456', 'ports': ['5432', '22', '53']}, '22']) => {
    "msg": "open port 22 for group sg-123456"
}
ok: [localhost] => (item=[{'group_id': 'sg-123456', 'ports': ['5432', '22', '53']}, '53']) => {
    "msg": "open port 53 for group sg-123456"
}
ok: [localhost] => (item=[{'group_id': 'sg-987654', 'ports': ['22']}, '22']) => {
    "msg": "open port 22 for group sg-987654"
}
ok: [localhost] => (item=[{'group_id': 'sg-18374923', 'ports': ['22', '3389']}, '22']) => {
    "msg": "open port 22 for group sg-18374923"
}
ok: [localhost] => (item=[{'group_id': 'sg-18374923', 'ports': ['22', '3389']}, '3389']) => {
    "msg": "open port 3389 for group sg-18374923"
}

PLAY RECAP *********************************************************************
localhost                  : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

Upvotes: 1

Related Questions