CODEWITHSUNDEEP
prometheusistiopromql
AngryPanda
AngryPanda

Reputation: 113

How to calculate requests per minute using Istio Prometheus metrics?

I want to calculate requests per minute, aggregated by service name.

I'm using the following query but I'm not sure if it is correct.

sum(increase(istio_requests_total{destination_workload_namespace="falabella"}[1m])) by (destination_workload)

Upvotes: 4

Views: 4751

Answers (2)

valyala
valyala

Reputation: 17794

It is more technically correct to use increase(istio_requests_total[1m]) instead of 60 * rate(istio_requests_total[1m]) for calculating the per-minute requests rate, since istio_requests_total is a counter with integer values (e.g. the total number of requests since the last counter reset). It is expected that increase() over the last minute must return integer values for integer counter, while rate() may return fractional values.

Unfortunately, Prometheus may return fractional results for increase() over integer counters because of extrapolation. See this issue for details. Additionally, increase() and rate() in Prometheus may miss counter increase for slow-changing counters - see this comment and this article for technical details. Prometheus developers are going to fix these issues - see this design doc. In the mean time it is possible to use VictoriaMetrics, which solves these issues in rate() and increase() functions from the start.

Upvotes: 2

Joel
Joel

Reputation: 2404

It looks correct. Another query would be:

60 * sum(rate(istio_requests_total{destination_workload_namespace="falabella"}[1m])) by (destination_workload)

As documented, they are equivalent: https://prometheus.io/docs/prometheus/latest/querying/functions/#increase

Upvotes: 5

Related Questions