external.metrics.k8s.io is not available in EKS cluster

Question

Kubernetes version : 1.13 on Amazon's EKS

I am trying to follow this guide https://nuvalence.io/building-a-k8s-autoscaler-with-custom-metrics/ to enable autoscaling based on rabbitmq data in our Kubernetes cluster by using the prometheus-adapter.

However, running kubectl get --raw /apis/external.metrics.k8s.io/v1beta1 | jq . returns Error from server (NotFound): the server could not find the requested resource which tells me that external metrics are not for some reason supported / enabled.

Custom ones do though, so kubectl get --raw /apis/custom.metrics.k8s.io/v1beta1 | jq . returns valid data.

The issue is that rabbitmq should definitively be an external metric and I spent hours trying to make it work as custom without any success.

No idea how to proceed with the external metrics or an indication about where to start searching? Should this api endpoint be there by default or there is an extension that enables it?

Answer 1

So it seems that external metrics are not coming out of the box, just by installing the prometheus-adapter helm chart, and this PR https://github.com/DirectXMan12/k8s-prometheus-adapter/pull/302 with a clear demonstration about using external metrics, helped me towards the solution.

Answer 2

external.metrics.k8s.io is not part of upstream kube API as mentioned in docs

For external metrics, this is the external.metrics.k8s.io API. It may be provided by the custom metrics adapters provided above.

So you need to install custom metric provider such as GCP stackdriver. I assume you are using EKS with prometheus, here is one good option that I used before prometheus-adapter. Kindly note that the API is v1beta1.custom.metrics.k8s.io though.

PS: I thought EKS 1.13 is already deprecated, you might need to update it first.