CODEWITHSUNDEEP
kuberneteskubeletkube-controller-manager
flyer
flyer

Reputation: 9826

Reasons of Pod Status Failed

If Pod's status is Failed, Kubernetes will try to create new Pods until it reaches terminated-pod-gc-threshold in kube-controller-manager. This will leave many Failed Pods in a cluster and need to be cleaned up.

Are there other reasons except Evicted that will cause Pod Failed?

Upvotes: 5

Views: 16848

Answers (3)

PjoterS
PjoterS

Reputation: 14102

PODs will not survive scheduling failures, node failures, or other evictions, such as lack of resources, or in the case of node maintenance. Pods should not be created manually but almost always via controllers like Deployments (self-healing, replication etc).

Reason why pod failed or was terminated can be obtain by

kubectl describe pod <pod_name>

Others situation I have encountered when pod Failed:

  • Issues with image (not existing anymore)
  • When pod is attempting to access i.e ConfigMap or Secrets but it is not found in namespace.
  • Liveness Probe Failure
  • Persistent Volume fails to mount
  • Validation Error

In addition, eviction is based on resources - EvictionPolicy

It can be also caused by DRAINing the Node/Pod. You can read about DRAIN here.

Upvotes: 2

Arbaaz
Arbaaz

Reputation: 981

There can be many causes for the POD status to be FAILED. You just need to check for problems(if there exists any) by running the command

kubectl -n <namespace> describe pod <pod-name>

Carefully check the EVENTS section where all the events those occurred during POD creation are listed. Hopefully you can pinpoint the cause of failure from there.

However there are several reasons for POD failure, some of them are the following:

  • Wrong image used for POD.
  • Wrong command/arguments are passed to the POD.
  • Kubelet failed to check POD liveliness(i.e., liveliness probe failed).
  • POD failed health check.
  • Problem in network CNI plugin (misconfiguration of CNI plugin used for networking).


For example:

pod failed due to image pull error

In the above example, the image "not-so-busybox" couldn't be pulled as it doesn't exist, so the pod FAILED to run. The pod status and events clearly describe the problem.

Upvotes: 6

Aashish Chaubey
Aashish Chaubey

Reputation: 597

Simply do this:

kubectl get pods <pod_name> -o yaml

And in the output, towards the end, you can see something like this:

enter image description here

This will give you a good idea of where exactly did the pod fail and what happened.

Upvotes: 4

Related Questions