Reputation: 11
I have an issue with displaying the metric value in my Slack template, no matter which convention I have tried to use it did not work, it is only working if I set it in the Annotations.Description, for instance:
{{ humanize $values.B.Value }}%
But when I’m trying to make it more dynamic so I can use it in all of my alerts instead of configuring each alert individually, it is not working, I’m trying to create a new value of “Current Value” which will display the metric value of the alerted instance, but for some reason, it is unable to extract the value of the metric, I have tried to declare it in many ways but nothing seems to work.
Here is the payload data of one alert that I have tried to configure for it a custom Annotation of the “Current Value” of the metric:
[
{
"annotations": {
"summary": "Instance instance1 has been down for more than 5 minutes"
},
"labels": {
"instance": "instance1"
},
"startsAt": "2024-03-13T06:41:09.128Z"
},
{
"annotations": {
"__alertId__": "50",
"__dashboardUid__": "ebRhY1b4z",
"__orgId__": "1",
"__panelId__": "2",
"__value_string__": "[ var='B' labels={DBInstanceIdentifier=node-0} value=25.516241395976735 ], [ var='C' labels={DBInstanceIdentifier=node-0} value=1 ]",
"__values__": "{\"B\":25.516241395976735,\"C\":1}",
"currentValue": "25.52%",
"description": "CPU Utilization is outside normal level"
},
"labels": {
"DBInstanceIdentifier": "node-0",
"__alert_rule_uid__": "ef630ac4-0b53-4f21-aa51-3bfcf11e9e60",
"__legacy_c_noc_test_alerts-slack__": "true",
"__legacy_use_channels__": "true",
"alertname": "AWS-DB (MySQL): CPU utilization alert",
"env": "dev",
"grafana_folder": "dev",
"rule_uid": "ef630ac4-0b53-4f21-aa51-3bfcf11e9e60"
},
"startsAt": "2024-03-14T09:26:00.000Z",
"endsAt": "2024-03-14T09:40:00.000Z"
}
]
One example of my template that I have tried to use:
{{ define "alert_severity_prefix_emoji" }}
{{- if ne .Status "firing" -}}:white_check_mark:
{{- else if eq .CommonLabels.severity "critical" -}}:red_circle:
{{- else if eq .CommonLabels.severity "warning" -}}:warning:
{{- end }}
{{- end }}
{{ define "slack.title" }}
{{- template "alert_severity_prefix_emoji" . }}[{{ .Status | toUpper }}{{ if eq .Status "firing" }} x {{ .Alerts.Firing | len }}{{ end }} [{{ .CommonLabels.env | toUpper }}] || {{ .CommonLabels.alertname -}}
{{- end }}
{{ define "slack.text" }}
{{ range .Alerts }}
{{ if .Annotations.summary }}*Summary:* {{ .Annotations.summary }}{{ end }}
{{ if .Annotations.description }}*Description:* {{ .Annotations.description }}{{ end }}
{{ if .Annotations.currentValue }}*Current Value:* `{{ .Annotations.currentValue }}`{{ end }}
*Labels:*
{{- range $key, $value := .Labels }}
• {{ $key }}: `{{ $value }}`
{{- end }}
{{ end }}
{{ end }}
I’m using Grafana v10.3.1
Thank you in advance!
Upvotes: 1
Views: 130