CODEWITHSUNDEEP
google-app-engine
Yarin
Yarin

Reputation: 183999

Lazy-loading large/complex model properties in Google App Engine

Let's say I'm modeling a website where a web page would be represented by a PageModel, like so:

class PageModel(db.Model):
    name = db.StringProperty()
    parentPage = db.SelfReferenceProperty()
    content = db.TextProperty()

I'd like to be able to pull a list of all my page objects, in order to render menus, etc., but without having to pull in the content for all the items. How would you model this object so that you could pull in the content only when you needed it? Would it require a 1-to-1 reference relationship with a separate 'content' model? And if so, would you make the reference on the page object or on the content object?

Upvotes: 0

Views: 255

Answers (1)

Bryce Cutt
Bryce Cutt

Reputation: 1525

You could move the content property into a new model (PageContentModel). I would implement the reference by having the parent of the PageContentModel be the PageModel (using the parent property of db.Model). This allows you to modify both of them in a single transaction (as they are in a single entity group).

One benefit of modeling things with the PageContentModel having a reference to the PageModel (as opposed to the PageModel having a reference to the PageContentModel) is that if you ever need content to be larger than 1MB you can do so by allowing each PageModel to have 1 or more PageContentModel objects and you would just split your content into 1MB chunks and write each chunk to a different PageContentModel instance. To be able to get the content back you would need the PageContentModel objects to have an "order" property associated with them so you can re-build your content in the correct order.

To query for the PageContentModel instances related to a PageModel you would use the ancestor filter like this:

PageContentModel.all().ancestor(page_model_instance)

As suggested by @Nick another way to do this would be to use the files api to write the content to a blob in the blobstore and then link that blob to the PageModel by having a BlobReferenceProperty on the PageModel. I have now had a chance to try this and it is working pretty well (despite it being an experimental feature). This would allow your content to be very large and, under the new pricing model, is actually cheaper than storing your content inside the datastore model.

Updated Feb 7, 2012 to include suggestion from @Nick about the blobstore.

Upvotes: 2

Related Questions