Horizontal auto scaler doesn't report CPU usage

Question

I am trying to test the Kubernetes HPA like in example here

kubectl run php-apache --image=gcr.io/google_containers/hpa-example --requests=cpu=200m --expose --port=80
kubectl autoscale deployment php-apache --cpu-percent=20 --min=1 --max=10
kubectl get hpa

output is

NAME         REFERENCE               TARGETS           MINPODS   MAXPODS   REPLICAS   AGE
php-apache   Deployment/php-apache   <unknown> / 20%   1         3         0          1h

Why isn't controller-manager able to get current usage?

---

Environment details :

• K8s version 1.7.3 with Heapster running with a service on port 80, in kube-system.
• InfluxDB is set as sink and I can see the usage/limit etc. correctly in Grafana as well as on Heapster API
• I can see the cpu requests (200m) on the deployed pod's yaml.
• No errors in controller manager logs. All I see is etcd watch and pod creation messages.
• There is no "horizontal-pod-autoscaler-sync-period" specified in controller manifest, default should be 30 seconds.
• "hpa describe" doesn't list any events ("<none>").
• Creating/deleting HPA doesn't create any log in controller-manager
• When I didn't have Heapster installed, controller-manager logs would complain. I do not see this message with Heapster installed : "

failed to compute desired number of replicas based on listed metrics for Deployment/xx/php-apache: failed to get cpu utilization: unable to get metrics for resource cpu: failed to get pod resource metrics: the server cannot complete the requested operation at this time, try again later (get services http:heapster:)

---

Edit 1 [14-Sep] :

• I have uploaded the Heapster logs here. No errors there.

• I checked the Heapster API directly for metrics, and I get response.

  curl http://10.107.55.59/api/v1/model/namespaces/cp/pods/php-apache-593471247-c65fh/metrics/cpu/usage

{ "metrics": [ { "timestamp": "2017-09-14T13:46:00Z", "value": 19008838 }, { "timestamp": "2017-09-14T13:47:00Z", "value": 19008838 }, { "timestamp": "2017-09-14T13:48:00Z", "value": 19008838 } ], "latestTimestamp": "2017-09-14T13:48:00Z" }

Answer 1

Validate if heapster is fetching data and giving metric, try to run below on kube-api server, replace namespace-name and pod-name in it. it will show metric, if all well, also paste heapster log here

curl http://localhost:8080/api/v1/model/namespaces/{namespace-name}/pods/{pod-name}/metrics/