CODEWITHSUNDEEP
kubernetes
Michael
Michael

Reputation: 1146

Are there issues with running user pods on a Kubernetes master node?

Many of the run-throughs for deploying Kubernetes master nodes suggest you use --register-schedulable=false to prevent user pods being scheduled to the master node (e.g. https://coreos.com/kubernetes/docs/latest/deploy-master.html). On a very small Kubernetes cluster it seems somewhat a wasteful of compute resources to effectively prevent an entire node from being used for pod scheduling unless absolutely essential.

The answer to this question (Will (can) Kubernetes run Docker containers on the master node(s)?) suggests that it is indeed possible to run user pods on a master node - but doesn't address whether there are any issues associated with allowing this.

The only information that I've been able to find to date that suggests there might be issues associated with allowing this is that it appears that pods on master nodes communicate insecurely (see http://kubernetes.io/docs/admin/master-node-communication/ and https://github.com/kubernetes/kubernetes/issues/13598). I assume that this would potentially allow a rogue pod running on a master node to access/hijack Kubernetes functionality not normally accessible to pods on non-master nodes. Probably not a big deal with if only running pods/containers developed internally - although I guess there's always the possibility of someone hacking access to a pod/container and thereby gaining access to the master node.

Does this sound like a viable potential risk associated with this scenario (allowing user pods to run on a Kubernetes master node)? Are there any other potential issues associated with such a setup?

Upvotes: 10

Views: 5578

Answers (3)

newcomers
newcomers

Reputation: 189

Now Kubernetes and some Kubernetes distribution have what it calls taint. taint can decide if the master can run a pod or not.

although running the pod on the master node is not the best practice but it's possible to do so. medium

in Kubernetes, we can read the explanation about taint here and I believe this is also related to scheduler

in Kubernetes or K3S, we can check if the nodes set the taint or not by describing the nodes.

# kubectl describe nodes | grep Taints 
Taints:             node.kubernetes.io/unreachable:NoExecute 
Taints:             node.kubernetes.io/unreachable:NoSchedule 
Taints:             node.kubernetes.io/unreachable:NoExecute 
Taints:             <none>

NoSchedule: Pods that do not tolerate this taint are not scheduled on the node.

PreferNoSchedule: Kubernetes avoids scheduling Pods that do not tolerate this taint onto the node.

NoExecute: Pod is evicted from the node if it is already running on the node, and is not scheduled onto the node if it is not yet running on the node. source

if you want to specify one of your nodes, rather master or agent, just mention the nodes

# kubectl describe nodes agent3 | grep Taints
  Taints:             <none> 
# kubectl describe nodes master | grep Taints
  Taints:             <none>

this is how you apply the taint to your nodes

kubectl taint nodes agent1 key1=value1:NoSchedule
kubectl taint nodes agent2 key1=value1:NoExecute

when your nodes are not running automatically it will show NoSchedule or NoExecute, make sure to check your nodes before checking the taint.

Upvotes: 0

AATHITH RAJENDRAN
AATHITH RAJENDRAN

Reputation: 5396

@robert have given a clear answer. I'm just trying to explain in a metaphorical way with a real-time example.

Your company's MANAGER is a better coder. If he starts coding, your company's MANAGER kind of work will be stalled/less efficient, because he can handle one thing in an efficient way. that will put your entire company at risks.

To operate efficiently, Hire more devs to code and don't make your MANAGER code(in order to get the works for the amount you are paying him).

Upvotes: -2

Robert Bailey
Robert Bailey

Reputation: 18230

Running pods on the master node is definitely possible.

The security risk you mention is one issue, but if you configure service accounts, it isn't actually much different for all deployed pods to have secure remote access to the apiserver vs. insecure local access.

Another issue is resource contention. If you run a rogue pod on your master node that disrupts the master components, it can destabilize your entire cluster. Clearly this is a concern for production deployments, but if you are looking to maximize utilization of a small number of nodes in a development / experimentation environment, then it should be fine to run a couple of extra pods on the master.

Finally, you need to make sure the master node has a sufficiently large pod cidr allocated to it. In some deployments, the master only gets a /30 which isn't going to allow you to run very many pods.

Upvotes: 9

Related Questions