Reputation: 863
I want to monitor irregular occurrences (a specific log row) and send an alert (from grafana) when that log row is seen more than X times in the last Y minutes.
Which metric should I use and how?
I only want to count the number of occurrences in the last Y minutes, the total number of occurrences "of all time" is not interesting.
Neither counters nor gauges seem to fit because that would mean that I have to reset the value manually to zero when I haven't seen the log row for some time. What I would like is a value that is 1 whenever I send a message to statsd and 0 otherwise so I can sum the 1s over the last Y minutes.
Upvotes: 0
Views: 268
Reputation: 718
No, counter will be reset to null after every flush (Default to 10 seconds).
So the problem I guess is that when you create the Grafana dashboard, you tell the Grafana that "Connect two points i+1 and i-1 into a line if the point i is a null point".
If my guess is right, you can check the configuration on "Display" -> "Null value" when you edit your Grafana dashboard and fix it.
Upvotes: 1