CODEWITHSUNDEEP
prometheusgrafanapromql
daniel0mullins
daniel0mullins

Reputation: 1959

Prometheus filtering when using a recording rule

I have the following recording rule set up in Prometheus:

- record: web:prod:inbound_request_duration_seconds:by_private_ip:rate1m
  expr: sum(rate(inbound_http_request_duration_seconds_sum{job="prod-web"}[1m])) by (private_ip)

And when I execute the query by specifying the rule name in Grafana (or Prometheus Explorer) the recording rule works just fine.

What I want to do is filter that recording rule by httpStatus which I am sending in the Histogram set up in my app.

If I run do something like this:

web:prod:inbound_request_duration_seconds:by_private_ip:rate1m{httpStatus="^5..$"}

I don't get an error, I just get no output, whereas if I take the raw query and do this:

sum(rate(inbound_http_request_duration_seconds_sum{job="prod-web", httpStatus="^5..$"}[1m])) by (private_ip)

I get the output I am expecting.

Is it possible to filter on a recording rule? Or do I have to create additional recording rules with the filters I want included?

Upvotes: 0

Views: 1172

Answers (1)

trallnag
trallnag

Reputation: 2376

Your recording rule aggregates all time series by private_ip and all other labels are removed. Use the following expression:

sum(rate(inbound_http_request_duration_seconds_sum{job="prod-web"}[1m])) by (private_ip, httpStatus)

Upvotes: 1

Related Questions