CODEWITHSUNDEEP
pythongoogle-app-engine
Raphaël Ambit
Raphaël Ambit

Reputation: 149

Instance memory limit exceeded with GAE. Go above 2,048 Mo?

I am curious regarding how to handle memory limits in GAE. Currently, I have this .py app that requires a lot of CPU / memory.

I tried to run it on GAE using b8 instances (basically the top instances with 4.8 Ghz CPU & 2,048 MO of memory). I have also allowed several instances in my app.yaml file (I tried to use up to 25 which is the max limit, just in case).

I also tried to set manually the number of CPU & memory to be used by the app as followed:

resources:
  cpu: 2
  memory_gb: 12.6
  disk_size_gb: 20
  volumes:
  - name: ramdisk1
    volume_type: tmpfs
    size_gb: 0.5

But whatever I do, I keep reaching the same memory limitation... (see below)

GET500 0 B 43 s Unknown /_ah/start Exceeded soft memory limit of 2048 MB with 3163 MB after servicing 0 requests total. Consider setting a larger instance class in app.yaml.

So here is my question: Is there an upper limit of 2048 whatever I try to do? Or maybe I am just not setting my app.yaml file properly and there is a way to push more memory in the process?

Note that I do realize this is quite a big app I am trying to launch, the code is already optimized, but it has to handle many operations on large datasets. Nevertheless, the app is meant to be ran punctually only.

Upvotes: 0

Views: 625

Answers (1)

Rafael Lemos
Rafael Lemos

Reputation: 5819

As mentioned by @gaefan in the comments you should consider using GAE flexible environment. The reason for that is that, as per the documentation, the standard environment limits the memory according to the class of the instance that you choose to use, in your case for the B8 it is 2048 MB.

This is not the case with the Flexible environment, as per the documentation, GAE takes care of assigning an instance with sufficient resources to at least guarantee what you have specify on your app.yaml file, so in your case, you would have your 12.6 GB specified

Upvotes: 1

Related Questions