CODEWITHSUNDEEP
pythongoogle-app-enginememory-leaks
Federico Capello
Federico Capello

Reputation: 1118

Google App Engine Instance Memory Error

I have a simple python App Engine application with low traffic (3 hours of instance activity per day more or less). I'm using everything below the free tier provided and I have:

The problem is that I get the Exceeded soft private memory limit of 128 MB with 131 MB after servicing 93 requests total error randomly once every 2/3 days but there is no request the app can't process singularly.

I've tried to call gc.collect() at the end of each request but the amount of memory keeps increasing event after servicing the request.

How can I solve this without upgrading to higher performing instances (F2)?

Thanks!

Upvotes: 0

Views: 491

Answers (1)

minou
minou

Reputation: 16553

I suspect you are using tasks to process batches of 400 entities. There is a weird quirk (I would call it a bug but Google doesn't seem to think so) with the way that GAE handles memory cleanup with tasks. When the task completes, GAE does not clean up the cache right away, and if you are doing tasks on a regular basis, this can add up.

My solution was to define some entities like this:

class MyEntity(ndb.Model):
    _use_cache = False
    _use_memcache = False

You can also disable caching in the get_multi() call itself.

For the 400 entities you are processing, if you disable caching, it might fix the problem for you.

Upvotes: 2

Related Questions