CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-run
scr4bble
scr4bble

Reputation: 309

What is happening to Cloud Run every day around 1:35 am UTC

What is happening almost every day to Cloud run around 1:35 AM UTC? It seems to hold the incoming requests for a while at this time and then it causes peak in number of requests when it releases them all at once to the containers. I only know that because in these peaks it exceeds number of DB connections allowed (different problem that I have already solved but here it is interesting because it shows that something is happening exactly at this time almost every day).

Number of requests coming to the service is constant throughout the time (all just periodic requests) so there is no peak caused from external services - also monitoring shows that when disruption happens, line displaying number of incoming requests is flat, but actual number of processed requests is peaking.

I cannot find anything in the documentation about such disruption to the service being possible.

Is it some kind of maintenance that moves/recreates the containers very slowly?

Upvotes: 0

Views: 141

Answers (1)

Andres S
Andres S

Reputation: 1247

You can look at the logs for those times to see if the instances are being restarted, The peak in requests could be caused by cold starts, specially if only one instance is running at the time, you can try using minimum instances to reduce the impact on these cold starts.

Cloud run instance restarts is expected behavior (for maintenance, updates, etc) the 'min-instances' flag specifies that there will be a designated amount of containers running at the given time, but does not require them to be the same containers along the way.

Upvotes: 1

Related Questions