CODEWITHSUNDEEP
prometheusgrafanapromql
Orville
Orville

Reputation: 37

Is there a possibility to execute a range function over an aggregation in PromQL?

Usually, having a range function and an aggregation in PromQL is constructed like this:

sum by(label) (increase(metric{label="label"}[1h]))

with sum being the aggregator and increase being the range function.

However, the metric that I'm trying to query only returns 1 at different times with different label values. It's basically an ever-increasing counter. The (stacked) graph in Grafana without any functions applied to it looks like this: graph So, applying a range function on this metric is useless, because it never changes (remains 1). I would like to sum by(label) first, and then execute increase over the result. It would look like something along the lines of this:

increase((sum by(label) (metric{label=~".*"}))[1h])

I tried doing this, naturally, looking at the documentation and experimenting with the syntax, to no avail. I even tried to use two seperate queries, which also wasn't very successful. I also tried the new query 'Builder' in Grafana 9 to see if that's possible, without success...

So, does anybody have a suggestion on how to apply a range function to an aggregation by(label)? Unfortunately, I can't directly change the data in Prometheus, and have to rely on the result of the query.

Upvotes: 0

Views: 1585

Answers (1)

star67
star67

Reputation: 1832

Prometheus Subqueries

It would look like something along the lines of this: increase((sum by(label) (metric{label=~".*"}))[1h])

The problem with this query is that you try to pass an instant vector to a function that accepts a range vector (see increase and range vs. instant vectors).

To get a range vector you need to make several queries, which is possible with either recording rules or subqueries:

increase((sum by(label) (metric{label=~".*"}))[1h:])

Count Aggregation

It's simple, the sum aggregation is used, but since you don't care about the value (always 1), consider using count (see aggregation-operators).

Wrong Data Model

Obviously, there is a problem with your data, that won't let you properly apply the counter functions (such as increase).

You have:

12:00
{label="blue"} 1

12:30
{label="blue"} 1
{label="red"} 1

13:00
{label="blue"} 1
{label="red"} 1
{label="green"} 1

While it should be:

12:00
{label="blue"} 1

12:30
{label="blue"} 2
{label="red"} 1

13:00
{label="blue"} 3
{label="red"} 2
{label="green"} 1

The current approach has many drawbacks, only a few of them:

  • waste of memory resources (N metrics with a value instead of a metric with value N)
  • waste of computation resources (additional query to "get a counter")
  • the result is not resistant to node restarts, since aggregation is applied before the counter function (increase) (see "rate then sum")

Upvotes: 3

Related Questions