CODEWITHSUNDEEP
ansible
Granolaboy
Granolaboy

Reputation: 333

Working with ansible's variables, lists and dicts

I want to deploy sensitive data using ansible-vault. Basically I want to iterate through every repository, iterate through its config files looking for secrets kept in the vault and replace them with the encrypted value in the vault. I want to pass the values of repositories as a variable to look through the secrets but keep getting errors.

I tried various approaches but I'm unsure how to work with these data structures.
Advice is very appreciated.

My code structure:

vars:
  repositories: ["service1", "service2"]
  service1:
    service1.conf: ["vault_user1", "vault_password1"]
    service1.env: ["vault_secret1"]
  service2:
    service2.conf: ["vault_user2", "vault_password2"]

- name: place secrets
  replace:
    path: "/tmp/{{ repositories }}/{{ item.key }}"
    regexp: "{{ item.value }}"
    replace: "{{ item.value }}"
  with_items: "{{ repositories }} | dict2items"

vault.yaml

vault_user1: a
vault_password1: b
vault_user2: c
vault_password1: d
vault_secret1: e

service1.conf

user=vault_user1
password=vault_password1
secret=vault_secret1

service2.conf

user=vault_user2
password=vault_password2

Upvotes: 0

Views: 524

Answers (1)

Vladimir Botka
Vladimir Botka

Reputation: 68024

Given the file with the variables (encrypted or not doesn't make any difference in this example) and the files (rooted in the local directory tmp in this example).

shell> cat vault.yaml
vault_user1: a
vault_password1: b
vault_user2: c
vault_password2: d
vault_secret1: e

shell> tree tmp
tmp
├── service1
│   ├── service1.conf
│   └── service1.env
└── service2
    └── service2.conf

shell> cat tmp/service1/service1.conf 
vault_user1
vault_password1
vault_secret1
vault_user2
vault_password2

shell> cat tmp/service1/service1.env 
vault_user1
vault_password1
vault_secret1
vault_user2
vault_password2

shell> cat tmp/service2/service2.conf 
vault_user1
vault_password1
vault_secret1
vault_user2
vault_password2

To solve the problem, there must be 2 nested loops. Iterate repositories in the first loop and a list of the variables in the second. This can be achieved by iterating include_tasks with the second loop inside the file. For example, create a file

shell> cat place_secrets.yml
- name: place secrets
  replace:
     path: "tmp/{{ dir }}/{{ item.0.key }}"
     regexp: "{{ item.1 }}"
     replace: "{{ my_vault[item.1] }}"
  with_subelements:
    - "{{ repo|dict2items }}"
    - value

In the playbook, read the variables into the dictionary my_vault, and iterate the list repositories

- hosts: localhost
  vars:
    repositories: [service1,service2]
    service1:
      service1.conf: [vault_user1, vault_password1]
      service1.env: [vault_secret1]
    service2:
      service2.conf: [vault_user2, vault_password2]

  tasks:
    - include_vars:
        file: vault.yaml
        name: my_vault
    - include_tasks: place_secrets.yml
      loop: "{{ repositories }}"
      loop_control:
        loop_var: outer_item
      vars:
        dir: "{{ outer_item }}"
        repo: "{{ lookup('vars', outer_item) }}"

gives

shell> cat tmp/service1/service1.conf 
a
b
vault_secret1
vault_user2
vault_password2

shell> cat tmp/service1/service1.env 
vault_user1
vault_password1
e
vault_user2
vault_password2

shell> cat tmp/service2/service2.conf 
vault_user1
vault_password1
vault_secret1
c
d
If the structure of the data can be simplified the playbook below gives the same results 
- hosts: localhost
  vars:
    repositories:
      service1.conf: [vault_user1, vault_password1]
      service1.env: [vault_secret1]
      service2.conf: [vault_user2, vault_password2]

  tasks:
    - include_vars:
        file: vault.yaml
        name: my_vault
    - name: place secrets
      replace:
         path: "tmp/{{ (item.0.key|splitext).0 }}/{{ item.0.key }}"
         regexp: "{{ item.1 }}"
         replace: "{{ my_vault[item.1] }}"
      with_subelements:
         - "{{ repositories|dict2items }}"
         - value

Upvotes: 1

Related Questions