How to have app engine avoid cold starts?

Question

Even when there are instances already running, I am still experiencing cold starts on some of the requests.

I thought that GAE would start some instances in the background and add them to the pool of active instances that serve requests only once the instances are started. Is that not the case? Is there a way to configure GAE to make it so?

Instead it seems like some of the requests are waiting the full duration of the new instance to be started, which can take up to 10 seconds, when using the existing instances only would have served all the benchmark traffic under a couple of seconds.

UPDATE: This is my app.yaml config:

runtime: nodejs10
env: standard
instance_class: F1
handlers:
  - url: '.*'
    script: auto
automatic_scaling:
  min_instances: 1
  max_instances: 3

Answer 1

What you're looking for are the Warmup requests:

Warmup requests are a specific type of loading request that load application code into an instance ahead of time, before any live requests are made. Manual or basic scaling instances do not receive an /_ah/warmup request.

And from Configuring warmup requests:

Loading your app's code to a new instance can result in loading requests. Loading requests can result in increased request latency for your users, but you can avoid this latency using warmup requests. Warmup requests load your app's code into a new instance before any live requests reach that instance.

Not 100% perfect - there are some limitations, but they're the next best thing.

Configuring warmup requests means:

• Enabling warmup requests in your app.yaml file:

  inbound_services:
  - warmup
  

• Creating your handler for the '/_ah/warmup' warmup requests URL