CODEWITHSUNDEEP
prometheus
Rakesh N
Rakesh N

Reputation: 2850

Prometheus Kubernetes SD Config - pod annotation present

I want Prometheus to only scrape pods in my Kubernetes cluster with a certain annotation. So, in the job configuration , I want to use __meta_kubernetes_pod_annotationpresent_<annotationname> : true as suggested here - https://prometheus.io/docs/prometheus/latest/configuration/configuration/#pod per Kubernetes SD configuration for Prometheus.

The problem is, I am unable to find any example for the same :(

A colleague suggested -

- job_name: 'k8sPodScrape'
      kubernetes_sd_configs:
      - role: pod
      relabel_configs:
      - source_labels: ['__meta_kubernetes_pod_annotationpresent_my_custom_annotation']
        action: 'keep'
        regex: 'true'

But I am not really sure if it works and how? Any pointers would help greatly.

Upvotes: 2

Views: 9680

Answers (1)

Bless
Bless

Reputation: 5370

You can achieve the same using __meta_kubernetes_pod_annotation_<annotationname>. For example, in prometheus.yml, you can have this config:

      - job_name: 'kubernetes-pods'
        kubernetes_sd_configs:
        - role: pod
        relabel_configs:
        - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
          action: keep
          regex: true

action: keep and regex: true ensures that metrics will be scraped from the pod only if the annotation prometheus.io/scrape is true.

This annotation should be added in your pod definition:

annotations:
  prometheus.io/scrape: 'true'

So now prometheus scrapes metrics from the pod only if prometheus.io/scrape is set to true.

Upvotes: 6

Related Questions