CODEWITHSUNDEEP
dockerswarm
Cabrinha
Cabrinha

Reputation: 495

Swarm rescheduling after adding new node

With the new version of Rancher is it possible to tell docker SWARM (1.12+) to redistribute containers when I add a new node in my infrastructure?

Suppose I have 4 nodes with 5 containers on each, if I add a 5th node, I'd like to redistribute my containers to have 4 of them on each node.

When a node crashes or it shuts down (scaling down my cluster), the re-scheduling triggers well, but when I scale up by adding 1 or more nodes, nothing happens.

Upvotes: 1

Views: 963

Answers (2)

nukalov
nukalov

Reputation: 1357

I was having the exact same issue. But rather than because of adding a new node, I had a node failure on the underling shared storage between nodes (was using shared NFS storage for sharing mount points of read only configs).

The docker service update --force $(docker service ls -q) as of version Docker version 17.05.0-ce, build 89658be does not work.

scale-up.sh:

#!/bin/bash

echo "Enter the amount by which you want to scale your services (NUMBER), followed by ENTER: "
read SCALENUM

for OUTPUT in $(docker service ls | awk '{print $2}' | sed -n '1!p')
do
    echo "Scaling up "$OUTPUT" to "$SCALENUM
    docker service scale $OUTPUT=$SCALENUM
done

scale-down.sh:

#!/bin/bash

for OUTPUT in $(docker service ls | awk '{print $2}' | sed -n '1!p')
do
    echo "Scaling down "$OUTPUT" to 0"
    docker service scale $OUTPUT=0
done

Note that the second script SCALES DOWN the service, making it unavailable. You can also use the following command as a starting point for other scripting you may need, as this prints out the service name independently of the other columns in a typical docker service ls command:

$(docker service ls | awk '{print $2}' | sed -n '1!p')

I hope this helps!

Upvotes: 0

MagicMicky
MagicMicky

Reputation: 3879

This is not currently possible to do this. What you can do, is update a service with docker service update (i.e.: by adding an environment variable)

A new feature coming in docker 1.13 will be a force update of services, that will update the service and force the redistribution of nodes, so something like docker service update --force $(docker service ls -q) might be possible (haven't tried this yet, so can't confirm yet).

You can find more info about this feature in this blogpost

Upvotes: 1

Related Questions