CODEWITHSUNDEEP
pythongoogle-app-enginejinja2
voscausa
voscausa

Reputation: 11706

loading Jinja templates from python modules (pre-compiled templates)

I'am using Python 2.5 on App Engine and tried to get the Jinja2 ModuleLoader to work.

To initialize the environment I use :

@staticmethod                                                   # get Jinja environment (global)
def get_new():                                                   # and initialize Jinja environment
    if  myEnv._my_env == None :
        path = os.path.join(os.path.dirname(__file__), 'compiled')
        myEnv._my_env = Environment(loader = ModuleLoader(path))     

    return myEnv._my_env

'compiled' is a directory in my GAE project. But I receive TemplateNotFound exceptions all the time??

I compiled the templates using :

    env = Environment(extensions=['jinja2.ext.i18n'])

    env.filters['euros'] = Euros

    db_runtimes = Runtimes.all()                                                       # the html templates saved in a db.Blob
    for each in db_runtimes.fetch(999) :
        key = each.key().name()
        source =  db.Blob(each.content).decode('utf-8')
        name = key.split('.')[0]
        raw = env.compile(source, name=name, filename=name + '.py', raw=True)
        each.compiled = db.Blob(raw.encode('utf-8'))                           # compile and save the .py
        each.put()

The resulting code looks fine. Any ideas? I hope you can help me. This article from Rodrigo Moraes shows that loading templates from python modules is very fast. But in this 2009 proof of concept he "hacked" the Jinja code to be able to run the code. I think the ModuleLoader should do the same job. https://groups.google.com/group/pocoo-libs/browse_thread/thread/748b0d2024f88f64

The testmod.py looks like this :

from __future__ import division
from jinja2.runtime import LoopContext, TemplateReference, Macro, Markup, TemplateRuntimeError, missing, concat, escape, markup_join, unicode_join, to_string, identity, TemplateNotFound
name = u'testmod.py'

def root(context, environment=environment):
    if 0: yield None
    yield u'<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0      Transitional//EN"\n"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml">\n<head>\n<meta http-equiv="Content-Type" content="text/html;   charset=utf-8" />\n<title>TEST</title>\n</head>\n<body>\n\t<p>test template</p>\n</body>\n</html>'

blocks = {}
debug_info = ''

And the page handler :

def get(self):

    my_env = myEnv.get() 
    page = 'testmod.py' 
    template = my_env.get_template(page)  
    self.response.out.write(template.render({}))

I'have also tried to get the template without the .py extension.

Upvotes: 4

Views: 3582

Answers (3)

voscausa
voscausa

Reputation: 11706

Update: See this Gist with the code I use for a CMS:

https://gist.github.com/voscausa/9154936

Update : Now I am using Python 27 and was able to create a module loader, which can load jinja compiled templates (python code) from a package or from the database (package = None).

To initialize the loader, you can use:

from jinja2 import Environment
Environment(auto_reload=False, loader = moduleloader.FileSystemModuleLoader(package))

class ModuleLoader(object):
    """Base mixin class for loaders that use pre-parsed Jinja2 templates stored
    as Python code.
    """
    def get_module(self, environment, template):
        raise TemplateNotFound(template)

    def load(self, environment, filename, j_globals=None):
        """Loads a pre-compiled template, stored as Python code in a template
        module.
        """
        if j_globals is None: j_globals = {'environment' : environment}

        t = object.__new__(environment.template_class)

        module = self.get_module(environment, filename)
        name, blocks, root, debug_info = module.run(environment, t)   # module run function      

        t.environment = environment
        t.globals = j_globals
        t.name = name
        t.filename = filename
        t.blocks = blocks
        # render function and module
        t.root_render_func = root
        t._module = None

        # debug and loader helpers
        t._debug_info = debug_info
        t._uptodate = lambda: True
        return t

class FileSystemModuleLoader(ModuleLoader):

    def __init__(self, package = None):   # load module from package or datastore
        self.package = package            # package = "compiled" or package = None

    def get_module(self, environment, template):
        # Convert the path to a module name
        name = template.replace('.html', '').replace('.txt','').replace('/', '.')   # NO extensions   
        module = None

        if self.package == None :
            if name in sys.modules : return sys.modules[name]
            logging.info('load module : ' + name)      # load module from runtimes db
            try :
                runtime = models.Runtimes.get_by_key_name(template)
                module_code = db.Text(runtime.compiled)                                 
                module = imp.new_module(name)
                exec module_code in module.__dict__                                         
                sys.modules[name] = module             # add to sys modules, so no import
                return module
            except (ImportError, AttributeError):
                logging.error('load failed : ' + name)

        else :                                         # load module from package
            logging.info('load module : ' + name + ' from package : ' + self.package)
            try: 
                mod_import = __import__(self.package, globals(), None, [str(name)])
                module = getattr(mod_import, name)
                return module
            except (ImportError, AttributeError):
                logging.error('load failed : ' + name)

        raise TemplateNotFound(template)

Upvotes: 1

voscausa
voscausa

Reputation: 11706

I gave up on the ModuleLoader and went back to the solution of Rodrigo Moraes. I did not have to "hack" the Jinja2 code. I choose to insert two lines of code in the generated source to get the environment.

from templating import myEnv
environment = myEnv.get()                  # get (and initialize) the Jinja Environment global

And I changed Rodrigo's load function in the ModuleLoader :

def load(self, environment, filename, j_globals=None):
    """Loads a pre-compiled template, stored as Python code in a template
    module.
    """
    if j_globals is None: j_globals = {'environment' : environment}

    t = object.__new__(environment.template_class)

    module = self.get_module(environment, filename)
    # name, blocks, root, debug_info = module.run(environment, t)    CHANGED THE HACK

    t.environment = environment
    t.globals = j_globals
    t.name = module.name
    t.filename = filename
    t.blocks = module.blocks

    # render function and module
    t.root_render_func = module.root
    t._module = None

    # debug and loader helpers
    t._debug_info = module.debug_info
    t._uptodate = lambda: True
    return t

And the results looks very promising.

Upvotes: 0

Tim Hoffman
Tim Hoffman

Reputation: 12986

I assume you a pre-compiling the templates in the dev environment then deploying them. Have you confirmed that the files are accessible in the deployed env ?

Upvotes: 0

Related Questions