How to get list of all variables in jinja 2 templates

Question

I am trying to get list of all variables and blocks in a template. I don't want to create my own parser to find variables. I tried using following snippet.

from jinja2 import Environment, PackageLoader
env = Environment(loader=PackageLoader('gummi', 'templates'))
template = env.get_template('chat.html')

template.blocks is dict where keys are blocks, how can I get all variables inside the blocks ?

Answer 1

I needed an answer for my Flask project and I used @Kracekumar's answer with a minor update, which is unique for Flask projects. I would like to post my answer to guide Flask user's who is looking for the same solution.

If you trying to access jinja templates, you can use the flask's jinja_env which is saves you from creating a Enviroment with PackageLoader or FileSystemLoader. This is better because if you have a structre like this and have different {extends ...} statements, configuring the correct environment can be challenging.

Here is the similar answer with using Flask's current_app.jinja_env which is more safe to use in Flask IMO:

Make sure you are in a valid app_context. If not use context manager to do the operations, see the flask's document here

from jinja2 import meta
from flask import current_app

env = current_app.jinja_env
template_source, _, _ = env.loader.get_source(env, 'page_content.html')
parsed_content = env.parse(template_source)
meta.find_undeclared_variables(parsed_content)

---

Versions:

Flask==2.3.2
Jinja2==3.1.2

Answer 2

Below is the sample code that can be used get properties of the vars like {{ something.nested }}, without using any third party library.

Code

from jinja2 import Environment, Template, meta
from jinja2.nodes import Getattr, Getitem

def get_placeholders_from_template(content: str):
    environment: Environment = Environment()
    parsed_content: Template = environment.parse(content)
    nodes = parsed_content.body[0].nodes
    parent_nodes = meta.find_undeclared_variables(parsed_content)
    attr_nodes = [node for node in nodes if isinstance(node, Getattr)]
    item_nodes = [node for node in nodes if isinstance(node, Getitem)]
    placeholders = []
    for node in attr_nodes:
        placeholders.append(f"{node.node.name}.{node.attr}")

    for node in item_nodes:
        placeholders.append(f"{node.node.name}({node.arg.value})")

    print("variables: ", parent_nodes)
    print("Node Placeholders: ", placeholders)

Input

content = ("Hello {{ customer.first_name }} {{ customer.last_name }}, "        
           "Appraisal Progress: {{ appraisal.progress }} JobID: {{ Job['1234'] }}")

get_placeholders_from_template()

Output

variables:  ['customer', 'appraisal', 'Job']
Node Placeholders:  ['customer.first_name', 'customer.last_name', 'appraisal.progress', 'Job(1234)']

Note:

This code can be modified to perform a recursive operation to fetch deeper nested values like {{ something.more.some_more.once_more }}

Answer 3

Though the top answer attaches a note supporting extends, actually, it's not. The env.loader.get_source can only parse the current template.

Here's an implementation for the advice provided by the Violet Shreve's comment in the top answer.

import re, os
from jinja2 import Environment, meta, FileSystemLoader

template_path = 'the path to your templates'
def parse_template_variables(template_name):
    res = set()
    env = Environment(loader=FileSystemLoader(template_path))
    # considering the including and the extending
    stack = [template_name]
    while len(stack) > 0:
        ref_template = stack.pop()
        source, filename, uptodate = env.loader.get_source(env, ref_template)
        parsed_content = env.parse(source)
        res = res.union(meta.find_undeclared_variables(parsed_content))
        # Recursively check if there are other dependencies
        for ref_temp_name in meta.find_referenced_templates(parsed_content):
        stack.append(ref_temp_name)
    return res

Answer 4

for ones want to find out all variable not just the variable need to be set from outside. one can do following:

from jinja2 import Environment, meta
from jinja2.nodes import Name
content='''
{% set y = 1%}
show y: {{ y }}
{% if x|default(2) == 1%}
{% endif %}
{{ z|default(3) }}
{{ w }}
{% set y2 = y3|default(6)%}
we do not use y2 at all
{% for _i in mylist %}
item is {{ _i }}
{% endfor %}
{{ dict1.x }}
{{ dict1.x2.y.z }}
'''
env = Environment()
parsed_content = env.parse(content)
print("undeclared=", meta.find_undeclared_variables(parsed_content))

vars = set()
vars_internal = set()
for name in parsed_content.find_all(Name):
    if name.ctx == 'load':
        vars.add(name.name)
    else:
        vars_internal.add(name.name)
print(f"{vars=}\n{vars_internal=}")
# we can see vars also contain the value that has been used
# y2 does not show up in vars because it never gets used
print(f"vars assigned but never used: {vars_internal.difference(vars)}")
print(f"vars need to be assigned from outside: {vars.difference(vars_internal)}")
print(f"vars only used internally: {vars_internal}")

output is:

undeclared= {'dict1', 'x', 'z', 'mylist', 'y3', 'w'}
vars={'_i', 'dict1', 'x', 'z', 'mylist', 'y', 'y3', 'w'}
vars_internal={'_i', 'y', 'y2'}
vars assigned but never used: {'y2'}
vars need to be assigned from outside: {'dict1', 'x', 'z', 'mylist', 'y3', 'w'}
vars only used internally: {'_i', 'y', 'y2'}

Someone is asking for the fields, it can also easily archived:

# to obtain the nested field:
def recurse_getattr(g: Getattr):
    if isinstance(g.node, Getattr):
        return recurse_getattr(g.node) + "." + g.attr
    return g.node.name + "." + g.attr

all_fields = set()
for g in parsed_content.find_all(Getattr):
    all_fields.add(recurse_getattr(g))
print(all_fields)
# will output {'dict1.x2.y', 'dict1.x', 'dict1.x2', 'dict1.x2.y.z'}

Answer 5

Why not regex?

If find it a lot easier to use regex:

import re
with open('templates/templatename.html') as f:
    variables = re.findall("\{\{\s(.*?)\s\}\}", f.read())

Answer 6

For me jinja2.meta.find_undeclared_variables(parsed_content) is not a good fit because it does not provide nested variables.

jinja2schema tool was kinda ok for simple scenarios but with all the loops and other jinja2 dark powers it was failing with errors.

I have played around with jinja2 data structures and was able to get all variables including nested ones. For my use case this was enough. Maybe this will also help for somebody else :)

Here is the code:

from jinja2 import Environment, FileSystemLoader, nodes


def get_variables(path, filename):
    template_variables = set()
    env = Environment(loader=FileSystemLoader(searchpath=path))
    template_source = env.loader.get_source(env, filename)[0]
    parsed_content = env.parse(template_source)
    if parsed_content.body and hasattr(parsed_content.body[0], 'nodes'):
        for variable in parsed_content.body[0].nodes:
            if type(variable) is nodes.Name or type(variable) is nodes.Getattr:
                parsed_variable = parse_jinja_variable(variable)
                if parsed_variable:
                    template_variables.add(parsed_variable)

    return template_variables


def parse_jinja_variable(variable, suffix=''):
    if type(variable) is nodes.Name:
        variable_key = join_keys(variable.name, suffix)
        return variable_key
    elif type(variable) is nodes.Getattr:
        return parse_jinja_variable(variable.node, join_keys(variable.attr, suffix))


def join_keys(parent_key, child_key):
    key = child_key if child_key else parent_key
    if parent_key and child_key:
        key = parent_key + '.' + key
    return key


if __name__ == "__main__":
    variable_keys = get_variables({replace_with_your_template directory}, {replace_with_your_template_file})
    print(*variable_keys, sep='\n')

Answer 7

Based on @Kracekumar's answer, but for the simplest use-case of just extracting tokens from a template passed as a string argument with no loading semantics or filter overrides:

env = jinja2.Environment()
parsed_content = env.parse(template_source)
tokens = jinja2.meta.find_undeclared_variables(parsed_content)

tokens will be a set.

Answer 8

Since no one has answered the question and I found the answer

from jinja2 import Environment, PackageLoader, meta
env = Environment(loader=PackageLoader('gummi', 'templates'))
template_source = env.loader.get_source(env, 'page_content.html')
parsed_content = env.parse(template_source)
meta.find_undeclared_variables(parsed_content)

This will yield list of undeclared variables since this is not executed at run time, it will yield list of all variables.

Note: This will yield html files which are included using include and extends.

Answer 9

For my pelican theme, i have created a tools for analyse all jinja variables in my templates files.

I share my code

This script generate a sample configuration from all variables exists in template files and get a variables from my official pelicanconf.py

The function that extract all variables from template file

def get_variables(filename):
    env = Environment(loader=FileSystemLoader('templates'))
    template_source = env.loader.get_source(env, filename)[0]
    parsed_content = env.parse(template_source)

The complete script

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# use:
# generate_pelicanconf-sample.py my_official_blog/pelicanconf.py

import sys
import imp
import os

from jinja2 import Environment, FileSystemLoader, meta


# Search all template files
def list_html_templates():
    dirList = os.listdir('templates')

    return dirList


# get all variable in template file
def get_variables(filename):
    env = Environment(loader=FileSystemLoader('templates'))
    template_source = env.loader.get_source(env, filename)[0]
    parsed_content = env.parse(template_source)

    return meta.find_undeclared_variables(parsed_content)


# Check if the pelicanconf.py is in param
if len(sys.argv) != 2:
    print("Please indicate the pelicanconf.py file")
    sys.exit()

# Get all vars from templates files
all_vars = set()
files = list_html_templates()
for fname in files:
    variables = get_variables(fname)
    for var in variables:
        if var.isupper():
            all_vars.add(var)

m = imp.load_source('pelicanconf', sys.argv[1])

# Show pelicanconf.py vars content
for var in all_vars:
    varname = 'm.%s' % var
    if var in m.__dict__:
        print ("%s = %s" % (var, repr(m.__dict__[var])))


    return meta.find_undeclared_variables(parsed_content)

The sample result of this program

LINKS = ((u'Home', u'/'), (u'archives', u'/archives.html'), (u'tags', u'/tags.html'), (u'A propos', u'http://bruno.adele.im'))
SITESUBTITLE = u'Une famille compl\xe8tement 633<'
DEFAULT_LANG = u'fr'
SITEURL = u'http://blog.jesuislibre.org'
AUTHOR = u'Bruno Adel\xe9'
SITENAME = u'Famille de geeks'
SOCIAL = ((u'adele', u'http://adele.im'), (u'feed', u'http://feeds.feedburner.com/FamilleDeGeek'), (u'twitter', u'http://twitter.com/jesuislibre.org'), (u'google+', u'https://plus.google.com/100723270029692582967'), (u'blog', u'http://blog.jesuislibre.org'), (u'facebook', u'http://www.facebook.com/bruno.adele'), (u'flickr', u'http://www.flickr.com/photos/b_adele'), (u'linkedin', u'http://fr.linkedin.com/in/brunoadele'))
FEED_DOMAIN = u'http://blog.jesuislibre.org'
FEED_ALL_ATOM = u'feed.atom'
DISQUS_SITENAME = u'blogdejesuislibreorg'
DEFAULT_PAGINATION = 10
GITHUB_BLOG_SITE = u'https://github.com/badele/blog.jesuislibre.org'

For more détail of this script see https://github.com/badele/pelican-theme-jesuislibre

Answer 10

I had the same need and I've written a tool called jinja2schema. It provides a heuristic algorithm for inferring types from Jinja2 templates and can also be used for getting a list of all template variables, including nested ones.

Here is a short example of doing that:

>>> import jinja2
>>> import jinja2schema
>>>
>>> template = '''
... {{ x }}
... {% for y in ys %}
...     {{ y.nested_field_1 }}
...     {{ y.nested_field_2 }}
... {% endfor %}
... '''
>>> variables = jinja2schema.infer(template)
>>>
>>> variables
{'x': <scalar>,
 'ys': [{'nested_field_1': <scalar>, 'nested_field_2': <scalar>}]}
>>>
>>> variables.keys()
['x', 'ys']
>>> variables['ys'].item.keys()
['nested_field_2', 'nested_field_1']