CODEWITHSUNDEEP
dockerdocker-swarm
Daniel Asanome
Daniel Asanome

Reputation: 498

Limit starting containers after complete shutdown

We have a bare metal Docker Swarm cluster, with a lot of containers.

And recently we have a full stop on the physical server.

The main problem, happened on Docker startup where all container tried to start on the same time.

I would like to know if there is a way to limit the amount of starting container? Or if there is another way to avoid overloading the physical server.

Upvotes: 3

Views: 83

Answers (1)

BMitch
BMitch

Reputation: 263489

At present, I'm not aware of an ability to limit how fast swarm mode will start containers. There is a todo entry to add an exponential backoff in the code and various open issues in swarmkit, e.g. 1201 that may eventually help with this scenario. Ideally, you would have an HA cluster with nodes spread in different AZ's, and when one node fails, the workload would migrate to another node and you do not end up with one overloaded node.

What you can use are resource constraints. You can configure each service with a minimum CPU and memory reservation. This would prevent swarm mode from scheduling more containers on a node than it could handle during a significant outage. The downside is that some services may go unscheduled during an outage and you cannot prioritize which are more important to schedule.

Upvotes: 3

Related Questions