CODEWITHSUNDEEP
pythongoogle-app-engineflask
BrainPermafrost
BrainPermafrost

Reputation: 674

How to diagnose Google App Engine Flask Memory Leak

I have a Google App Engine app running (Flask app) that seems to have a memory leak. See the plot of the memory usage below. The memory usage continually creeps up until it hits the limit and the instance is shutdown and a new one is started up.

It's a simple API with about 8 endpoints. None of them handle large amounts of data.

I added an endpoint that takes a memory snapshot with the tracemalloc package, and compares it to the last snapshot and then writes the output to Google Cloud Storage.

I don't see anything in the reports that indicates a memory leak. The peak memory usage is recorded as about 0.12 GiB.

I am also calling gc.collect() at the end of every function that is called by each endpoint.

Any ideas on how to diagnose this, or what might be causing it?

enter image description here

Upvotes: 0

Views: 894

Answers (1)

Andrei Tigau
Andrei Tigau

Reputation: 2048

There could be many reasons for this situation to be encountered. Is your app creating temporary files? Temporary files can be a cause of a memory leak. Temporary files can also be created from errors, or warnings. First of all, I would check my Stackdriver logs for errors and warnings and I would try to fix them.

Is your application interacting with databases or storage buckets ? Some memory related issues can be related to a bad interaction of your app with any data storage service. This issue was also encountered here and was mitigated by treating the Google Cloud Storage errors.

Another thing that you can do is investigate a bit the way of the memory is used in your function. For this you have some nice tools you can use like Pympler and Heapy. Playing with those tools may give you valuable clues about what your issue is.

Upvotes: 2

Related Questions