CODEWITHSUNDEEP
prometheuspromql
sdc
sdc

Reputation: 3011

How to divide two by prometheus queries to calculate a percentage

I have two distinct metrics that both have a label of client_app.

One metric is a histogram that counts the number of failed requests for a given time span (i.e. 10 req in the last min failed) and the other metric is a counter that increments for every request.

I want to divide these two metrics to get a percentage of failed requests per client_app

Here is my attempt so far

avg by (client_app) (max(rate(ignored_events_sum[5m])) / sum(rate(total_app_events[5m])))

This only outputs a single graph whereas I am hoping for one per each client_app

Upvotes: 4

Views: 4906

Answers (2)

valyala
valyala

Reputation: 17794

By default Prometheus performs division over pairs of time series with identical sets of labels on the left and the right side of / according to these docs. If time series on the left and the right sides of / contain distinct sets of labels, then on() and group_left() modifiers may help:

  • The on() modifier limits labels, which are used for searching for matching time series pairs. For example a / on(client_app) b would search for time series pairs from a and b with identical client_app label only.

  • The group_left() modifier allows matching multiple time series with the given labelset on the left side of / with a single time series on the right side of /.

See more details about these modifiers in these docs.

So the solution to the original question is the following PromQL query:

rate(ignored_events_count[5m])
  / on (client_app) group_left()
rate(total_app_events[5m])

It will divide per-seconds rates for each time series with ignored_events_count name by per-second rates for time series with total_app_events name with the matching client_app label value.

Upvotes: 1

Marcelo Ávila de Oliveira
Marcelo Ávila de Oliveira

Reputation: 22311

Try this:

max by (client_app) (rate(ignored_events_sum[5m])) / sum by (client_app) (rate(total_app_events[5m]))

Upvotes: 6

Related Questions