CODEWITHSUNDEEP
prometheusprometheus-alertmanager
Abdul Rahuman Seeni Mohamed
Abdul Rahuman Seeni Mohamed

Reputation: 43

Prometheus Alertmanager custom alert message

Is there any way to change or add fields to Prometheus AlertManager's default alert json format

I'm using prometheus AlertManager to send alerts and also using webhooks to send alerts using HTTP POST (https://prometheus.io/docs/alerting/configuration/#wechat_config). Prometheus Alertmanager generate default json format for alerts but i need to change the json format (simply need to put the default json inside one key "records" or simply need to add a field in the default json message). Is it possible to add new key fields in the json?.

Default alert json format

{
  "version": "4",
  "groupKey": <string>,    // key identifying the group of alerts (e.g. to deduplicate)
  "status": "<resolved|firing>",
  "receiver": <string>,
  "groupLabels": <object>,
  "commonLabels": <object>,
  "commonAnnotations": <object>,
  "externalURL": <string>,  // backlink to the Alertmanager.
  "alerts": [
    {
      "status": "<resolved|firing>",
      "labels": <object>,
      "annotations": <object>,
      "startsAt": "<rfc3339>",
      "endsAt": "<rfc3339>",
      "generatorURL": <string> // identifies the entity that caused the alert
    },
    ...
  ]
}

Need to add new fields

{
  "records":[
{ "value":

...

}
]
}

Example json (added "records and value" to json)

{
  "records":[
{ "value":
{
  "receiver": "email-logs",
  "status": "firing",
  "alerts": [
    {
      "status": "firing",
      "labels": {
        "alertname": "CPUUsageAbove20%",
        "instance": "node-exporter:9100",
        "job": "node-exporter",
        "monitor": "my-project",
        "severity": "warn",
        "team": "raptors"
      },
      "annotations": {
        "dashboard": "www.prometheus.io",
        "description": "CPU usage on node-exporter:9100 has reached 60"
      },
      "startsAt": "2020-04-30T08:03:17.309164516Z",
      "endsAt": "0001-01-01T00:00:00Z",
      "generatorURL": "http://0ab6d9955c65:9090/graph?g0.expr=60+%3E+job%3Anode_cpu_seconds%3Ausage+%3E+20\u0026g0.tab=1",
      "fingerprint": "9c558a8c20c2ba08"
    }
  ],
  "groupLabels": {
    "alertname": "CPUUsageAbove20%",
    "team": "raptors"
  },
  "commonLabels": {
    "alertname": "CPUUsageAbove20%",
    "instance": "node-exporter:9100",
    "job": "node-exporter",
    "monitor": "my-project",
    "severity": "warn",
    "team": "raptors"
  },
  "commonAnnotations": {
    "dashboard": "www.prometheus.io",
    "description": "CPU usage on node-exporter:9100 has reached 60"
  },
  "externalURL": "http://5493399b56dc:9093",
  "version": "4",
  "groupKey": "{}/{team=~\"^(?:(raptors|leafs))$\"}:{alertname=\"CPUUsageAbove20%\", team=\"raptors\"}"
}
}
]}

Upvotes: 0

Views: 6245

Answers (1)

Philip
Philip

Reputation: 1

Recently, I also came across this problem. After reading a lot of articles, my conclusion is this.

Providing the use of webhook as a generic receiver. The only way to manipulate JSON is to write your own integrations.

What I did is refer to some existing integrations like https://gitlab.com/yakshaving.art/alertsnitch, instead of writing data to DB, you can manipulate the data and then post to the HTTP API.

But you have to learn and write go code, as what I've done recently.

Upvotes: 0

Related Questions