CODEWITHSUNDEEP
pythonperformancegoogle-app-engine
d4nt
d4nt

Reputation: 15789

Performance issue when reading 100s of rows from AppEngine datastore using ancestor on development server.

I have a poorly performing query in my Google AppEngine app. Here's what I'm doing:

import models
from time import time

parent = models.ParentEntity.gql("WHERE string_property = :1", "a guid").get()

start = time()

data_rows = []

children = models.ChildEntity.gql("WHERE ANCESTOR IS :1", parent.key())
for child in children.run():
    data_rows.append(child.data)

print "Loaded %s rows in: %s" % (len(data_rows), time() - start)

I ran this three times in the interactive console on my machine, here is the output:

Loaded 653 rows in: 8.18052411079
Loaded 653 rows in: 5.93607997894
Loaded 653 rows in: 9.97486710548

Is 5-9 seconds to load 653 rows normal? This was on the dev web server, will it be much better on the real server? I know about memcache and am using it already, but how can I get things to go faster for the times I do need to populate the cache?

Upvotes: 1

Views: 100

Answers (1)

Daniel Roseman
Daniel Roseman

Reputation: 599778

You shouldn't take the dev server as any way indicative of performance on the live server. It emulates the BigTable infrastructure using either a flat file or an sqlite3 database, so performance is pretty terrible. The only way to test this properly is on AppEngine itself.

Note that an ancestor query is as efficient as it gets, so you shouldn't have any problems when you do try it in production.

Upvotes: 2

Related Questions