AppEngine starts new instances without need for them

Question

I am a subscriber for "Always On" option on Google AppEngine. I do not understand why does GAE have to start new instances when there are already "Always On" instances that are turned on and that do not have big number of requests at that time.

This drives me crazy since it nullifies what I get with the AlwaysOn subscription. Any ideas how to fix this behavior? I have latency problems because of this which I wanted to remove by using AlwaysOn. I am thinking of cancelling subscription since it just does not work.

Answer 1

I'd just like to add answer given to me on identical question I posted on google-app-engine group (Link). It helped me to lower the probability that new nodes are created when some active one exist at that time.

Do you have "threadsafe" set to true? If you do not, each instance will only handle a single request at a time.

It helped me a lot - give it a try if you have same issue as I did. Using_Concurrent_Requests

Answer 2

I don't think this necessarily nullifies the benefit of Always On, which, presumably, is that users should less frequently have to wait for a new instance to spin up to serve their request.

If you have Warmup Requests enabled, App Engine may spin up new instances in an anticipatory fashion even if your persistent instances can handle the current load. In this circumstance, the presence of a dynamic instance does not imply that a user's request was blocked waiting for the instance to load.

I would suggest adding enough logging to tell whether the dynamic instances are actually servicing user requests on startup. If so, what are they doing? Perhaps you have a single dynamic page that has to call several additional dynamic handlers in order to render all the page assets. This could be causing the unexpected traffic spike.

Answer 3

The only thing AlwaysOn does is ensure that there is always some instance to serve your requests, even if your application has endured a prolonged time without any traffic. At least to my knowledge, it doesn't change anything when it comes to the scaling algorithm App Engine uses to determine whether a need of new instance arises.

Unfortunately, little is known about the details of this behavior of GAE (i.e. deciding when to start new instance). It might be that your application had a lot of traffic recently, a lot of traffic yesterday at the same time of day, lot of traffic last week on the same weekday at the same time, etc. Either of these factors (and most likely many more unknown ones) might have impact on number of instances GAE delegates for your application.

I can sympathize with your complaints about latency issues but in my experience the AlwaysOn feature reduced them by quite a lot. Nevertheless, one must remember that low latency is not the priority of GAE as a service - it's the ability to handle gigantic traffic consistently without the need of explicit scaling.