gotothesky
gotothesky

Reputation: 111

Distinguish each alert, but they all come out red

I want to mark Critical Alert in red, Warning Alert in orange, and Normal Alert in green. But when all three alerts are sent at the same time, they all come out in red. I want each Slack Alert notification color to be distinguished. By the way, they all come out in red as shown in the picture below.

Here is my Grafana Alert Template.

{{ define "slack_title" }}
  {{ if gt (len .Alerts.Firing) 0 }}
  šŸ”„ {{ len .Alerts.Firing }} alert(s) firing
  {{ end }}
  {{ if gt (len .Alerts.Resolved) 0 }}
  āœ… {{ len .Alerts.Resolved }} alert(s) resolved
  {{ end }}
{{ end }}

{{ define "slack_message" }}
  {{ if gt (len .Alerts.Firing) 0 }}
    {{ range .Alerts.Firing }} {{ template "slack_alert_firing" .}} {{ end }} {{ end }}
  {{ if gt (len .Alerts.Resolved) 0 }}
    {{ range .Alerts.Resolved }} {{ template "slack_alert_resolved" .}} {{ end }} {{ end }}
{{ end }}

{{/* alert */}}
{{ define "slack_alert_firing" }}
šŸ”„ Alert Fired
Cluster: {{ .Labels.Cluster }}
Instance: {{ reReplaceAll ":9100" "" .Labels.instance }}
Value: {{ .Annotations.Value }}
Node: {{ reReplaceAll "aks-" "" .Labels.nodename }}
Severity: {{ .Labels.severity }}
{{ end }}

{{ define "slack_alert_resolved" }}
āœ… Alert Resolved
Cluster: {{ .Labels.Cluster }}
Instance: {{ reReplaceAll ":9100" "" .Labels.instance }}
Value: {{ .Annotations.Value }}
Node: {{ reReplaceAll "aks-" "" .Labels.nodename }}
Severity: Normal
{{ end }}

{{/* Color of Slack attachment (appears as line next to alert )*/}}
{{ define "slack.monzo.color" -}}
    {{ if eq .Status "firing" -}}
        {{ if eq .CommonLabels.severity "warning" -}}
            warning
        {{- else if eq .CommonLabels.severity "critical" -}}
            danger
        {{- else -}}
            #439FE0
        {{- end -}}
    {{ else -}}
    good
    {{- end }}
{{- end }}

enter image description here

Upvotes: 0

Views: 308

Answers (1)

Rusowsky 0x
Rusowsky 0x

Reputation: 1

idk if you were able to solve the issue, but my approach would be to use notification policies to fire different

by doing so, you would send several messages simultaneously, each of them with its corresponding color.

https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/notification-policies/

Upvotes: 0

Related Questions