CODEWITHSUNDEEP
pythonperformancejsongoogle-app-enginegoogle-cloud-datastore
shafty
shafty

Reputation: 605

Python/Google app engine data storage performance

In a python/google app engine app, I've got a choice between storing some static data (couple KB in size) in a local json/xml file or putting it into the datastore and querying it from there. The data is created by me, so there's no issues with badly formed data. In specific terms such as saving quota, less resource usage and app speed, which is the better method for this case?

I'd guess using simplyjson to read from a json file would be better since this method doesn't require a datastore query, whilst still being reasonably quick.

Taking this further, the app doesn't require a large data store (currently ~400KB), so would it be worthwhile moving all the data to a json file to get around the quota restrictions?

Upvotes: 1

Views: 507

Answers (3)

Mani Jagadeesan
Mani Jagadeesan

Reputation: 24275

For superior app performance, as Chris and others pointed out, python dict is the best.

But if you are ok with the minimal performance hit caused by datastore queries, I think that is the way to go purely from a design and maintenance perspective. Simplicity takes precedence over performance if you are not approaching the quota limits.

I assume yours is a simple app as of today. But it can gradually grow in complexity. As you add more features, having a hard-coded data somewhere will come in the way of design flexibility even in the short term. And you might end up rewriting those areas in future.

Upvotes: 1

Chris Farmiloe
Chris Farmiloe

Reputation: 14185

If your data is small, static and infrequently changed, you'll get the best performance by just writing your data as a dict in it's own module and just import it where you need it. This would take advantage of the fact that Python will cache your modules on import.

Upvotes: 5

Cameron
Cameron

Reputation: 98886

It is faster to keep your data in a static file instead of the datastore. As you said, this saves on datastore quota, and also saves time in round-trips to the datastore.

However, any data you store in static files is static and cannot be changed by your application (see the "Sandbox" section here). You would need to re-deploy the file each time you wanted to make changes (i.e. none of the changes could be made through a web interface). If this is acceptable, then use this method, since it's simple and fast.

Upvotes: 0

Related Questions