How to limit instance spawning (autoscaling) on GAE?

Question

I have noticed a recent surge in instance spawning on GAE.

In my app.yaml I have clearly defined that max two instances should be created at a time.

application: xxx
version: 1-6-0
runtime: python27
api_version: 1
instance_class: F2
automatic_scaling:
  max_idle_instances: 2
threadsafe: true

However the dashboard is showing 4 instances and the bills are going up. How can I stop this madness? :)

Answer 1

its possible that some of your requests are taking too long to complete, and thus cause new instances to be spawned. You could work around this (I'm told, but have yet to try it myself) by setting a high value for your min_pending_latency property. this could hurt your latency a little, but would also limit the rate of instance-spawning.

Answer 2

I did a lot of research into this.

F instances are automatically scaled. There is no way to limit that. Hence it makes sense to move the actual work away from frontend instances and put that into a backend instance (B1 or B2). The latter provides another 8 hours free quota.

The real challenge is to re-architect the app to use a default app.yaml for web statics, a mobile.yaml for mobile requests with shorter min_pending_latency and a backend.yaml (B2) instance for handling the tasks and calculations.

All this needs to be routed properly via a dispatch.yaml. In this file you can specify which url endpoint will be handled effectively by which module.

Best way to understand it, is to look at this excellent example from GAE:

It makes sense trying to make it work first on local environment, before trying anything on remote server.

dev_appserver.py dispatch.yaml app.yaml mobile.yaml backend.yaml

Also this official documentation explains some of the above in more detail.

Its pretty impressive, what can be achieved with GAE.

Answer 3

max_idle_instances

The maximum number of IDLE instances that App Engine should maintain for this version.

It seems that it is not currently possible to set the max number of instances for automatic scaling module. As @DoIT suggests, you can set spending limit, however, keep in mind the below.

When an application exceeds its daily spending limit, any operation whose free quota has been exhausted fails.

So if you need to control somehow the total number of instances and keep your service running, I see the following possibilities.

1. Change your scaling type to basic and set max_instances parameter as you like

2. Keep automatic scaling type and increase min_pending_latency and max_concurrent_requests parameters (multi-threading has to be enabled)

   You can find more details here.

Answer 4

The '''max_idle_instances''' set the max number of idle instances, i.e. Instances that are waiting for traffic spike. From your screenshot, it looks like all instances are getting traffic, so it looks ok to me. You can set max daily budget, if you want to control your spend on GAE.