Call alerts integration with Prometheus Alertmanager

Question

im setting Prometheus as a monitoring system, with Alertmanager. As an alert i need service that will call on selected number or group of numbers.

Best solution for us would be not to have an 3rd party provider who is gonna be covering this for us.

Thanks a lot.

Answer 1

I made a webhook that does this:

This is a program that will receive webhooks from prometheus alertmanager to send them as phone calls via twilio.

alert2twcall

Answer 2

The most popular VoIP provider is Twilio, which offers a Text2Speech API. When the phone is picked up, the text you send as a parameter (for example, "Alarm: SiteA.com is down") will be spoken. Next, you'll need to specify the URL of your script in the Webhook field for the Prometheus alert. A similar API can be used for SMS alerts.

A simpler solution is to use the service Pushcall.me (if you do not require detailed alerts). We use it for health-check failures. You can simply copy the call generation webhook URL from the website and paste it wherever you need. The key here is that picking up the phone is not necessary; you enter 1-5 numbers directly into the phone contact book and by the fact of the call, you understand what the alert is about. For example: +123xxxxxxxx (SiteA.com Alarm), +124xxxxxxxx (SiteB.com Alarm).

screenshot: example from Grafana alert manager

Answer 3

Prometheus doesn't call phone numbers. (Nor does it send SMS, for that matter.) You need to either write your own webhook handler (i.e. an HTTP server) that then makes an API call to a 3rd party provider.

The other option is to hook up your Alertmanager to a service like Opsgenie/Pagerduty that includes phone calls as an option to get notifications.

That being said, I don't think any of these options will be free (as in beer).