CODEWITHSUNDEEP
kubernetesamazon-ec2
박성일
박성일

Reputation: 143

metrics-service in kubernetes not working

I'm running kubernetes using an ec2 machine on aws. Node is in Ubuntu.

my metrics-server version.

wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml

components.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: metrics-server
  namespace: kube-system
  labels:
     k8s-app: metrics-server
  spec:
       serviceAccountName: metrics-server
  volumes:
  # mount in tmp so we can safely use from-scratch images and/or read-only containers
  - name: tmp-dir
    emptyDir: {}
  containers:
  - name: metrics-server
    image: k8s.gcr.io/metrics-server/metrics-server:v0.3.7
    imagePullPolicy: IfNotPresent
    args:
      - --cert-dir=/tmp
      - --secure-port=4443
      - --kubelet-preferred-address-type=InternalIP,ExternalIP,Hostname
      - --kubelet-insecure-tls

Even after adding args, the error appears. error : Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)

or

error: metrics not available yet

No matter how long I wait, that error appears.

my kops version : Version 1.18.0 (git-698bf974d8)

i use networking calico.

please help...

++ I try to wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.5.0/components.yaml

view logs..

kubectl logs -n kube-system deploy/metrics-server

"Failed to scrape node" err="GET "https://172.20.51.226:10250/stats/summary?only_cpu_and_memory=true": bad status code "401 Unauthorized"" node="ip-172-20-51-226.ap-northeast-2.compute.internal"

"Failed probe" probe="metric-storage-ready" err="not metrics to serve"

Upvotes: 7

Views: 14136

Answers (2)

Antonio
Antonio

Reputation: 51

Last comment is useful.You can edit the deploy directly as well and adding line "--kubelet-insecure-tls=true" its enought for me:

Edit deploy:

$ kubectl edit deployment.apps/metrics-server -n kube-system

Add the line:

- --kubelet-insecure-tls=true

Similar result:

  containers:
  - args:
    - --cert-dir=/tmp
    - --secure-port=4443
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --kubelet-use-node-status-port
    - --metric-resolution=15s
    - --kubelet-insecure-tls=true

And save with ":wq" and enjoy.

~$ kubectl top pods -n kube-system
NAME                                  CPU(cores)   MEMORY(bytes)
coredns-6d4b75cb6d-k8dmc              3m           18Mi
coredns-6d4b75cb6d-wxxn6              3m           17Mi
kube-apiserver-k8s-master1            82m          306Mi
kube-apiserver-k8s-master2            65m          247Mi
kube-controller-manager-k8s-master1   32m          47Mi
kube-controller-manager-k8s-master2   4m           19Mi
kube-proxy-9dbgk                      1m           9Mi
kube-proxy-bwhdm                      1m           14Mi
kube-proxy-fz8v8                      1m           15Mi
kube-proxy-vcnrc                      1m           9Mi
kube-scheduler-k8s-master1            7m           18Mi
kube-scheduler-k8s-master2            4m           16Mi
metrics-server-79576f7ff-97tpc        6m           15Mi
metrics-server-79576f7ff-qzczp        4m           13Mi

~$ kubectl top nodes
NAME          CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
k8s-master1   318m         15%    1047Mi          55%
k8s-master2   208m         10%    1002Mi          52%
k8s-worker1   30m          3%     804Mi           42%
k8s-worker2   35m          3%     550Mi           29%

Upvotes: 5

Sysad85
Sysad85

Reputation: 361

Download the components.yaml file manually:

wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

Then edit the args section under Deployment:

spec:
  containers:
  - args:
    - --cert-dir=/tmp
    - --secure-port=443
    - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
    - --kubelet-use-node-status-port
    - --metric-resolution=15s

add there two more lines:

    - --kubelet-insecure-tls=true
    - --kubelet-preferred-address-types=InternalIP

kubelet Of 10250 The port uses https agreement , The connection needs to be verified by tls certificate. Adding ,--kubelet-insecure-tls tells it do not verify client certificate.

After this modification just apply the manifest:

kubectl apply -f components.yaml

wait a minute and you will see metrics server pod is up

Upvotes: 15

Related Questions