CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastore
shapeare
shapeare

Reputation: 4233

How to do view N more comments with Google App Engine Python API

In Facebook, if the comments under a topic is too many, there will be a link "view more comments", and if click the link, it will load more comments.

  1. How to design the model classes?

I designed the model class like:

class Item(db.Model):
      ...
      comments = db.ListProperty(str)

Or should I design a separate class for the comments and add reference to the item, such that

class Comment(db.Model):
      ...
      item = db.ReferenceProperty(Item)
  1. How to do this with Jinja?

Currently, I do this with a for loop to iterate over the comments.

Upvotes: 1

Views: 69

Answers (1)

stevep
stevep

Reputation: 959

You have some exposure here to the 1 second entity update if two comments arrive nearly simultaneously and instance spiking. (If you do go this route, be sure you set indexed=False for you list property.) Consider how this plays out before proceeding vs. an alternative that seems more complex, but may offer real advantages once you are in production.

Why not create a CommentParent model with a counter field, and a separate Comments model with the parent_id and date_created (auto) as indexed fields. When a new comment comes in that starts a new comments thread put() the parent entity, and send the comment text and parent_id reference to a taskqueue. All future comments to the thread are similarly just passed-thru to the queue (yielding very fast on-line handler latency). Have the taskqueue put() the Comments records, and update the CommentParent counter. To avoid the 1 sec limit on the counter update, you need to think a bit about either regulating your TQ, or subtituting a sharded counter for the entity counter. (Counters in very high-volume applications are notoriously hard.)

When a user enters into this comment thread, the client receives the parent_id and the the current count. It then requests the first N records which is handled (assumed LIFO model) with a simple query using parent_id= date_created> filters, and ordered by date_created descending and either a fetch() limit, or a counter to limit your processing inside a query.ier() loop (I would use the latter.) You pass back the comments, and each comments date_created (to be your "cursor pointer" for next query). Client uses the initial counter value to maintain the "load N more" button label. (I would also add another return value which is an is_finished boolean set to true anytime your query returns fewer than you fetch limit - this will avoid full reliance on the counter math.)

There are some downsides to this. Eventual consistency and taskqueue delays may mean occasional out-of-sync counts, or missed comments. You have overhead related to the custom index needed for the date_created g.t. and order descending setup. However, you also have real positives: a very fast latency for your on-line handler, and no issues with a cluster of near-simlutaneous comments causing you issues with the 1 second entity update limit. You also have a very important benefit of having the taskqueue act as a buffer for any small spikes in comment volumes. Without this buffer, you may be plagued by having a few instances started up each time there is such a spike. That's 15 minutes charged times number of such instances, (or else you are fiddling constantly with your instance Admin settings and complaining about instances fired up for little apparent need).

You also have some pretty easy ways to use memcache to limit the number of actual queries that need to be run. If a thread is active, you could cache the first N query results, and use that cache until a new comment is made. Things like that. Course you can also use memcache in your approach. No matter which, give some thought to using memcache.

HTH -stevep

Upvotes: 1

Related Questions