CODEWITHSUNDEEP
flaskjinja2
Drock
Drock

Reputation: 3

Flask Custom jinja2 extension caching template states after new request

I'm Looking to create a similar custom extension to push any javascript code blocks to a designated area on the page or below footer.

My version works using Python 3.6,Flask and Jinja 2.9. However I have a major issue that occurs after changing the line number or content within the blocks. The content will appear multiple times on render.

from jinja2 import nodes
from jinja2.ext import Extension

class JavascriptBuilderExtension(Extension):
    tags = set(['push'])

    def __init__(self, environment):
        super(JavascriptBuilderExtension, self).__init__(environment)
        self._myScope = {}
        environment.extend(
            pull = self._myScope
            )
    def parse(self, parser):
        """Parse tokens """
        tag = parser.stream.__next__()
        args = [parser.parse_expression(), nodes.Const(tag.lineno)]
        body = parser.parse_statements(['name:endpush'], drop_needle=True)
        callback = self.call_method('compiled', args)
        return nodes.CallBlock(callback,[], [], body).set_lineno(tag.lineno)

    def compiled(self,tagname,linenum,caller):
        tagname = "{}_{}".format( tagname, linenum)
        self._myScope[tagname] = caller()
        return "<!-- moved {} from line {} -->".format(tagname,linenum)

My template code looks like this

<html> <head></head> <body> <h1>Test template</h1>
{% push 'js' %} X {% endpush %}
{% push 'html' %} Z {% endpush %}
{% push 'js' %} Y {% endpush %}
{{ pull }}
</body> </html>

My rendered output is below:

<html> <head></head> <body> <h1>Test template</h1>
name = hyper testing jinja
date = right now
<!-- moved js_4 from line 4 -->
<!-- moved html_5 from line 5 -->
<!-- moved js_6 from line 6 -->
{'js_4': ' X ', 'html_5': ' Z ', 'js_6': ' Y '}
</body> </html>

The Problem happens after I change the template block line number or content.

After changing content and line numbers

<html> <head></head> <body> <h1>Test template</h1>
{% push 'js' %} ABC {% endpush %}

{% push 'html' %} Z {% endpush %}

{% push 'js' %} 123{% endpush %}
{{ pull }}
</body> </html>

Render changed blocks now has prior content

<html> <head></head> <body> <h1>Test template</h1>
name = hyper testing jinja
date = right now
<!-- moved js_4 from line 4 -->
<!-- moved html_7 from line 7 -->
<!-- moved js_9 from line 9 -->
{'js_4': ' X ABC', 'html_5': ' Z ', 'js_6': ' Y ','js_9':'123','html_7':'Z'}
</body> </html>

This issue causes duplicate content to be added into the response.

Is there a way to call the extension on ever page request to re-parse the template for new changes? or Possible to not cache the enclosed extension blocks?

I have already tried adding the code below to auto reload templates but does not help the issue.

app.jinja_env.auto_reload = True

Update: Added link to test code

Jinja custom extension test code

It appears that calling render_template_string doesn't cache and renders properly when changes are made. Not sure why render_template method caches.

Upvotes: 0

Views: 1740

Answers (1)

Tarun Lalwani
Tarun Lalwani

Reputation: 146510

Okie, so this took a lot of hours to figure out, because there were two issues.

Flask doesn't reload templates when extension are added

env = app.jinja_env
env.add_extension("flaskext.JavascriptBuilderExtension")

When you add the extension like above, then jinja_env gets created even before the first request. This internally check if TEMPLATES_AUTO_RELOAD is set or not, if it is not set then value of debug is checked. But as of now even our app.run(debug=True) has not been called. So that is why templating reloading doesn't get enabled

Solution is to add config manually before accessing the jinja_env

app.config['TEMPLATES_AUTO_RELOAD'] = True
app.config['EXPLAIN_TEMPLATE_LOADING'] = True
env = app.jinja_env
env.add_extension("flaskext.JavascriptBuilderExtension")

Using request context instead of global context

Next the Extension is only initialized once. So you have used below code

def __init__(self, environment):
    super(JavascriptBuilderExtension, self).__init__(environment)
    self._myScope = {}
    environment.extend(
        pull = self._myScope
        )

The _myScope variable is created at extension level which will remain there till flask is running. So you are creating a variable and sharing data across any page render even with different request. The need is to use context that is only live till the request is. For this one can use jinja2.nodes.ContextReference

Also since the data is now available only in context we need to use {% pull %} instead of {{ pull }}. I could not inject a variable into context from the extension. There might be a way but my experimentation around that failed. So below is the final class I used

from jinja2 import nodes
from jinja2.ext import Extension
from jinja2.nodes import ContextReference


class JavascriptBuilderExtension(Extension):
    tags = set(['push','pull'])

    def __init__(self, environment):
        super(JavascriptBuilderExtension, self).__init__(environment)
        self._myScope = {}

    def parse(self, parser):
        raise NotImplementedError()

    def preprocess(self, source, name, filename=None):
        return super(JavascriptBuilderExtension, self).preprocess(source, name, filename)

    def parse(self, parser):
        """Parse tokens """
        tag = parser.stream.__next__()
        ctx_ref = ContextReference()

        if tag.value == "push":
            args = [ctx_ref, parser.parse_expression(), nodes.Const(tag.lineno)]
            body = parser.parse_statements(['name:endpush'], drop_needle=True)
            callback = self.call_method('compiled', args)
        else:
            body = []
            callback = self.call_method('scope', [ctx_ref])

        return nodes.CallBlock(callback, [], [], body).set_lineno(tag.lineno)

    def scope(self, context, caller):
        return str(context.vars["_myScope"])

    def compiled(self, context, tagname, linenum, caller):
        tagname = "{}_{}".format(tagname, linenum)

        if "_myScope" not in context.vars:
            context.vars["_myScope"] = {}

        context.vars["_myScope"][tagname] = caller()

        return "<!-- moved {} from line {} -->".format(tagname, linenum)

Key changes were - Get the context using ctx_ref = ContextReference() - Pass it in the callback args - Create _myScope in context if one doesn't exist - Make a separate render for pull

Now when I make the request the html generated is

<html> <head></head> <body> <h1>Test template</h1>
<!-- moved js_2 from line 2 -->

<!-- moved html_4 from line 4 -->

<!-- moved js_6 from line 6 -->

{'js_6': Markup(u' DEF '), 'html_4': Markup(u' Z '), 'js_2': Markup(u' ABC ')}
</body> </html>

Upvotes: 2

Related Questions