CODEWITHSUNDEEP
prometheus-alertmanagerprometheus-operator
sjm
sjm

Reputation: 465

Silencing Alerts on Multiple AlertManagers Simultaneously

we are running a large number of kubernetes clusters in our network and each with its own prometheus-operator deployment. Every deployment has its own AlertManager deployment. We are finding it very time consuming to silence an alert across all the clusters.

Currently what we have to do is to go to individual alertManager and silence the Individual alert there.

What we are hoping to achieve is an easy way of silencing the alerts for all the cluster (ideally from a single GUI)

Don't want to use inhibit as it kills the purpose of alert.

Any one has any idea how to do that?

Upvotes: 1

Views: 636

Answers (1)

Xtigyro
Xtigyro

Reputation: 421

The most obvious way should be - by turning off the deployment of an Alertmanager per K8s cluster and using a single central Alertmanager cluster where the firing alerts would be silenced.

A concern might be the performance of that single Alertmanager if there is a very big number of K8s clusters. That requires testing.

Upvotes: 1

Related Questions