CODEWITHSUNDEEP
pythongoogle-app-engine
Christophe Debove
Christophe Debove

Reputation: 6296

Increase retrieving perf python appengine with memcache

Hello I want to know if in the appengine env the things I did is usefull Because I don't know why this page is very slow.

class Foo(db.Model):
    id = db.StringProperty(multiline=True)
    name = db.StringProperty(multiline=True)
    date = db.DateTimeProperty(auto_now_add=True)
    description = db.TextProperty()    
    carac = db.StringProperty(multiline=True)



class FoosPage(webapp.RequestHandler):
  def get(self):
    foos =  memcache.get("all-foos")
    if foos is None:
        foos = Foo.all()
        size = foo.count()
    else :
        size = len(foo)

    foosTab=[]
    for i in range(size) :
        foosTab.insert(i,foos[i])
        memcache.set("all-foos", foosTab, 60)
    template_values = {
        'foos': foos
            }
    path = os.path.join(os.path.dirname(__file__) + '/../templates/', 'foos.html')
    self.response.out.write(template.render(path, template_values))

Upvotes: 2

Views: 142

Answers (2)

Nick Johnson
Nick Johnson

Reputation: 101139

This block of code:

foos =  memcache.get("all-foos")
if foos is None:
    foos = Foo.all()
    size = foo.count()
else :
    size = len(foo)

foosTab=[]
for i in range(size) :
    foosTab.insert(i,foos[i])
    memcache.set("all-foos", foosTab, 60)

Can be replaced with this (much simpler) code:

foos = memcache.get("all-foos")
if not foos:
  foos = Foo.all.fetch(1000) # At most 1000 foos
  memcache.set("all-foos", foos, 60)

Specifically, it avoids the unnecessary call to count() (which issues an expensive RPC for something you'll find out anyway when you fetch the result), it does a single fetch call instead of iterating over the results and fetching in batches of 20, and it only calls memcache set (once!) if it had to fetch the list in the first place.

Upvotes: 2

Dave W. Smith
Dave W. Smith

Reputation: 24966

You have the memcache.set() inside the loop. That's a lot of unnecessary traffic to the memcache service. Do it once, after the loop.

Further, the way you have that coded, there's no need for establishing size.

foosTab = []
for foo in foos:
    foosTab.append(foo)

or, more idiomatically

foosTab = [foo for foo in foos]

That'll save you doing a separate count().

Upvotes: 5

Related Questions