Include variables into a playbook from an in-place rendered jinja2 template without intermediate file

Question

I need to create some variables, lists, etc that are dynamically generated from other variables -- think of a list of nodes for kubernetes built with for loop.

This can be done inside Ansible with the set_fact and looping through incrementally with a default(), adding each list item each loop, but I don't want to build up all these vars with tasks.

I want to use a template to build the list and then use the result as variables. This works fine if I render the template to a file, and then load it into the playbook with vars_files or include_vars, but I wonder if it is possible to skip the intermediate file and do an in-place render-template realize-variables?

rendered_template.j2

nodes:
{% for item in nodes_struct %}
  - name: "{{ item.name }}"
    ipaddress: "{{ item.ipaddress }}"
    networkifacename1: "enp1s0"
    networkifacename2: "enp2s0"
{% endfor %}

playbook.yaml

- hosts: all
  vars_files:
    - ../vars/rendered_template.yaml

or

- name: Render global variables
  include_vars: "../vars/rendered_template.yaml"

The above is just a very simple example from a snippet inside the template. Most of the solutions suggest building up the vars instead with playbook tasks using the map function or nodes: | construct. I was trying to avoid doing ~60 playbook tasks instead of 60 vars rendered from the template in-place, but maybe that's the only way.

Would have been nice if I could use something like

- name: show templating results
  set_fact:
    template_var: "{{ lookup('template', '../vars/rendered_template.j2') }}"

And then somehow flatten it into var space from the root of that rendered template var instead of getting to it like. code below does not work)

- debug:
    msg: "{{ template_var.nodes }}"

Answer 1

Ok, I went elsewhere and coaxed the answer to my Q. This code works for my purpose and is concise.

- set_fact:
    template_var: "{{ lookup('template', '../vars/rendered_template.j2') }}"

- set_fact:
    template_var_yaml: "{{ template_var | from_yaml }}"

- name: Convert to top-level
  set_fact:
    "{{ item.key }}": "{{ item.value }}"
  loop: "{{ template_var_yaml|dict2items }}"
  loop_control:
    label: "{{ item.key }}"

- debug:
    var: nodes

Of course, it can be reduced down to

- name: Convert to top-level
  set_fact:
    "{{ item.key }}": "{{ item.value }}"
  loop: "{{ lookup('template', '../vars/rendered_template.j2')|from_yaml|dict2items }}"
  loop_control:
    label: "{{ item.key }}"

- debug:
    var: nodes

Leaving the vars "namespaced" from inside template_var.nodes would also be useful so as not to pollute the top-level var space, but in my case, the source template variables were used extensively later by a bunch of roles I'm using that someone else made, so I need to keep them top-level.

Answer 2

The iteration in set_fact or in a template can always be used when there are no filters or functions to create the structure in a pipe. But in your case, the template is overkill. Instead, you can put the common attributes into a dictionary

  node_default:
    networkifacename1: enp1s0
    networkifacename2: enp2s0

and declare the list in the pipe

  nodes: "{{ nodes_struct|product([node_default])|map('combine') }}"

gives what you want

  nodes:
  - ipaddress: 1.1.1.1
    name: foo
    networkifacename1: enp1s0
    networkifacename2: enp2s0
  - ipaddress: 2.2.2.2
    name: bar
    networkifacename1: enp1s0
    networkifacename2: enp2s0

---

• Example of a complete playbook for testing

- hosts: localhost

  vars:

    nodes_struct:
      - name: foo
        ipaddress: 1.1.1.1
      - name: bar
        ipaddress: 2.2.2.2

    node_default:
      networkifacename1: enp1s0
      networkifacename2: enp2s0

    nodes: "{{ nodes_struct|product([node_default])|map('combine') }}"

  tasks:

    - debug:
        var: nodes|to_nice_json

gives the same in JSON

  nodes:
    [
        {
            "ipaddress": "1.1.1.1",
            "name": "foo",
            "networkifacename1": "enp1s0",
            "networkifacename2": "enp2s0"
        },
        {
            "ipaddress": "2.2.2.2",
            "name": "bar",
            "networkifacename1": "enp1s0",
            "networkifacename2": "enp2s0"
        }
    ]

---

• You can put the declarations into your current vars files

shell> cat vars/rendered_template.yaml 
node_default:
  networkifacename1: enp1s0
  networkifacename2: enp2s0
nodes: "{{ nodes_struct|product([node_default])|map('combine') }}"

The play below gives the same result

- hosts: localhost

  vars:

    nodes_struct:
      - name: foo
        ipaddress: 1.1.1.1
      - name: bar
        ipaddress: 2.2.2.2

  tasks:

    - include_vars: rendered_template.yaml
    - debug:
        var: nodes|to_nice_json

See: Resolving local relative path why you can omit vars from the path vars/rendered_template.yaml

---

• You can simplify it if you still want to use the template

nodes: |
  {% filter from_yaml %}
  {% for item in nodes_struct %}
  - {{ item|combine(node_default) }}
  {% endfor %}
  {% endfilter %}

Answer 3

You're already most of the way there. You can do this:

nodes: |
  {% filter from_yaml %}
  {% for item in nodes_struct %}
  - name: "{{ item.name }}"
    ipaddress: "{{ item.ipaddress }}"
    networkifacename1: "enp1s0"
    networkifacename2: "enp2s0"
  {% endfor %}
  {% endfilter %}

Here's a runnable example:

- hosts: localhost
  gather_facts: false
  vars:
    nodes_struct:
      - name: foo
        ipaddress: 1.1.1.1
      - name: bar
        ipaddress: 2.2.2.2

    nodes: |
      {% filter from_yaml %}
      {% for item in nodes_struct %}
      - name: "{{ item.name }}"
        ipaddress: "{{ item.ipaddress }}"
        networkifacename1: "enp1s0"
        networkifacename2: "enp2s0"
      {% endfor %}
      {% endfilter %}
  tasks:
    - debug:
        msg: "{{ nodes }}"

Running that playbook produces:

PLAY [localhost] ***************************************************************

TASK [debug] *******************************************************************
ok: [localhost] => {
    "msg": [
        {
            "ipaddress": "1.1.1.1",
            "name": "foo",
            "networkifacename1": "enp1s0",
            "networkifacename2": "enp2s0"
        },
        {
            "ipaddress": "2.2.2.2",
            "name": "bar",
            "networkifacename1": "enp1s0",
            "networkifacename2": "enp2s0"
        }
    ]
}

PLAY RECAP *********************************************************************
localhost                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0