CODEWITHSUNDEEP
kubernetesprometheusprometheus-alertmanager
user389955
user389955

Reputation: 10457

How to define a variables in Kubernetes's PrometheusRule

My current PrometheusRule at Kubernetes is implemented as follows: (note there is a hardcoded number 10000000000:

---
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  namespace: mytest
  labels:
    team: myteam
  name: myname
spec:
  groups:
  - name: mygroup
    rules:
    - alert: MemoryUsage
      annotations:
        message: MemoryUsage at ec2 {{ $labels.cluster_id }} is {{printf "%.2f" $value}}%.  Threshold is 80%.
      expr: 100 * memory_average{container="XXX"...} / 10000000000 > 80
      for: 20m
      labels:
        severity: warning

Question 1: How can I define a variable to replace the  hardcoded number 10 000 000 000? I.e. Where and how to define something like this in PromethusRole: max_memory=10 000 000 000 , and then replace the expr to something like this?

expr: 100 * memory_average{container="XXX"...} / var.max_memory > 80

Question2: I want the value of max_memory different for different clusters. In PromethusRole, how can I define the variable with the logic as below? Thank you!

if  {{ $labels.cluster_id }} == "ClusterA":
    max_memory = 10000000000
else If  {{ $labels.cluster_id }} == "ClusterB":
    max_memory = 7000000000

Upvotes: 0

Views: 1255

Answers (2)

zer0
zer0

Reputation: 2899

You are talking about templatizing your manifests. You will need a template system for this which will generate these YAMLs for you before you can apply them to your cluster.

There are multiple ways to achieve this, you can try Helm / mustache, or kustomize to begin with. These will allow you to create templates in the way you are describing, and when you generate your manifests, these values will be substituted and you will have ready manifests you can apply to your cluster.

For more advanced applications you can look into jinja template system, which is Python based. This is also the system ansible uses to generate the playbook manifests.

Upvotes: 1

RyanDotnet
RyanDotnet

Reputation: 144

1) How can I define a variable to replace the hardcoded number 10 000 000 000? I.e. Where and how to define something like this in PromethusRole: max_memory=10 000 000 000 , and then replace the expr to something like this?

You can use something like below (if you are using Helm charts and getting the values from your Values.yaml file)

expr: 100 * memory_average{container="XXX"...} / var.max_memory > {{ .Values.memoryThresholdValue}}

Values.Yaml

memoryThresholdValue : anyValue

2) I want the value of max_memory different for different clusters. In PromethusRole, how can I define the variable with the logic as below?

Here also you can use {{ .Values.maxvalue}}, if you variable is defined in the helm template file the use something like below -

expr: max_over_time(reloader_last_reload_successful{namespace=~".+", ingress= {{ include "kube-prometheus-stack.operator.fullname" . }} }[5m]) == {{ .Values.nginxValue }}

Upvotes: 1

Related Questions