David Tinker
David Tinker

Reputation: 9624

Is it possible to promote a Kubernetes worker node to master?

Is it possible to promote a Kubernetes worker node to master to quickly recover from the loss of a master (1 of 3) and restore safety to the cluster? Preferably without disrupting all the pods already running on it. Bare metal deployment. Tx.

Upvotes: 2

Views: 2400

Answers (1)

David Tinker
David Tinker

Reputation: 9624

It doesn't look like a worker node can be promoted to master in general. However it is easy to sort out for a specific case:

  • Control plane node disappears from the network
  • Node is manually deleted: k delete node2.example.com --ignore-daemonsets --delete-local-data
  • Some time later it reboots and rejoins the cluster

Check that it has rejoined the etcd cluster:

# k exec -it etcd-node1.example.com -n kube-system -- /bin/sh
# etcdctl --endpoints 127.0.0.1:2379 --cacert /etc/kubernetes/pki/etcd/ca.crt \
    --cert /etc/kubernetes/pki/etcd/server.crt --key /etc/kubernetes/pki/etcd/server.key \
    member list
506801cdae97607b, started, node1.example.com, https://65.21.128.36:2380, https://xxx:2379, false
8051adea81dc4c6a, started, node2.example.com, https://95.217.56.177:2380, https://xxx:2379, false
ccd32aaf544c8ef9, started, node3.example.com, https://65.21.121.254:2380, https://xxx:2379, false

If it is part of the cluster then re-label it:

k label node node2.example.com node-role.kubernetes.io/control-plane=
k label node node2.example.com node-role.kubernetes.io/master=

Upvotes: 2

Related Questions