CODEWITHSUNDEEP
kubernetes
Contraboy
Contraboy

Reputation: 129

Why are K8 pod limits not being applied?

I am basically trying to apply pod CPU limits on my cluster. I made the edit to my deployment.yaml (added requests, limits) in there and applied the yaml. Below are the observations:

  1. I don't get any error when I apply the file (my existing app pod is terminated, a new one is spun up)
  2. I can see a QoS class being applied when I do a describe on my pod (Qos: Burstable)
  3. The issue is that the limits are not being honored, as I can see in my metrics server that the pod CPU > 300 (whereas the limit set is 200m)
  4. I have a istio sidecar container attached to the pod (but I only want to apply the limit to my app an not Istio)

Snippet of yaml file:

resources:
    limits:
        cpu: 200m
        memory: 100Mi
    requests:
        cpu: 50m
        memory: 50Mi

Any ideas what else I need to check here, I checked all documentation and get no errors but the limits are not being applied. Thanks in advance!

Upvotes: 1

Views: 1107

Answers (1)

Yaron Idan
Yaron Idan

Reputation: 6765

Pod CPU includes all containers in the pod, while the limits you've specified apply only to the app container.
If you query metrics for the container alone, you will probably find that it's honouring the limits you've enforced upon it.
Here's an example prometheus query you can use if you're running it on your cluster, that will allow you to understand the ratio between every container actual CPU usage and it's CPU requests -

max(sum(irate(container_cpu_usage_seconds_total{container=~"<container_name>", image !="",namespace="<namespace>", pod=~"<Deployment_name>"}[5m])) by (pod, namespace)) 
/ 
max(sum(kube_pod_container_resource_requests_cpu_cores{namespace="<namespace>", container=~"<container_name>", pod=~"<deployment_name>"}) by (pod,namespace))

Upvotes: 4

Related Questions