CODEWITHSUNDEEP
pythongoogle-app-enginefilesystemslocalhost
Katedral Pillon
Katedral Pillon

Reputation: 14864

How to create files at runtime on python app-engine: live or localhost

The question is simply as stated in the title. But the reason is probably important for a good answer.

I need to generate static html files at runtime. Generating the files is not the problem. Using jinja2 and webapp2 I can painlessly generate dynamic html files at runtime and serve them to my users. My problem is that generating these files is expensive. So I want to be able to save them as static html files; so that, when user tries to access them, I just serve the static html if it exists. If it does not exist, then I can create the file and then serve it.

Again I am already using jinja2 to create my string (i.e. file content). So my question is, how do I save the file to a path that my app.yaml file can map to?

Also if you are thinking memcache I am already using that. It is not sufficient storage. So while it’s a buffer, it’s not as good as having static html files.

Some back story:

If I can generate the files in localhost, that'd be even better. The thing is the site has a huge number of pages. But the structure is common to all the pages. So we store the relevant data in the datastore and use jinja2 to inflate the different pages. But because of heavy usage, memcache is not able to keep up. So now it's looking more cost-effective to create static html pages from the data stored in the datastore. Creating those pages by hand is obscene and error prone. So if we could generate the html pages as usual (i.e. jinja2 template and datastore) and then have an automated system for creating the html pages, it would be awesome. We could do the app.yaml part manually.

So after I do 

template = jinja_environment.get_template(‘template.html')
content = template.render(template_values)

What do I do to save the file to say ./relevant/path/filename.html

Upvotes: 0

Views: 220

Answers (2)

GAEfan
GAEfan

Reputation: 11360

IfI understand your question properly, the best way to do this would be to store your static files in Google Cloud Storage (GCS). Easy to read and write to GCS. And easy to delete the file when you no longer need it.

Upvotes: 2

Mike
Mike

Reputation: 1246

You can't write to files in App Engine -- see https://cloud.google.com/appengine/kb/java?csw=1#writefile for explanation.

Why don't you store the post-rendered page content back in datastore? I'm assuming the "expensive" part of the approach you're using now is the building of the templated content from the data. Once you've done that, instead of writing it off to local files, you could either put it as an additional property on the original data entity or you could create an entirely new model just for the rendered pages.

Essentially this would be a durable datastore implementation instead of memcache, if the issue is that pages are getting evicted too often from memcache.

A single fetch (directly from the key, since you can use keynames that you can create based on the page you're fetching, so you don't have to even do a query) is lightweight. Then, if you're still using memcache, then you'd check memcache first; if not there, you fetch the entity from the datastore and check to see if the rendered property contains data. If not, you use the data in the entity to render it with Jinja, store it back off in the rendered property, put the entity, store it off in memcache, and return the rendered content back to the client.

Upvotes: 0

Related Questions