CODEWITHSUNDEEP
prometheusgrafana
MorRich
MorRich

Reputation: 446

How to sum metrics in Prometheus across jobs

I have two jobs configured in promethus.yml

- job_name: serviceA
  scrape_interval: 60s
  metrics_path: /
  static_configs:
  - targets:
    - serviceA:8080
- job_name: serviceB
  scrape_interval: 60s
  metrics_path: /
  static_configs:
  - targets:
    - serviceB:8080

Both services have a counter metric named in each metric1 in serviceA; and metric2 in serviceB.

In Grafana and the Prometheus site the expression metric1 + metric2 does not return anything. I tried multiplication, division, etc but no results either. And same result with gauge metric as well.

What am I doing wrong here?

Upvotes: 1

Views: 5119

Answers (1)

Marcelo Ávila de Oliveira
Marcelo Ávila de Oliveira

Reputation: 22321

The metrics must have the same labels and values.

If "metric1" has labels "labelA" and "labelB", and "metric2" has labels "labelA" and "labelC", "metric1 + metric2" will return nothing.

If "metric1" has label "labelA" with "A" and "B" values, and "metric2" also has label "labelA" but with "C" and "D" values, "metric1 + metric2" will return nothing.

You can try to use functions to aggregate common labels and values, for example:

sum by (label1) (metric1) + sum by (label1) (metric1)

Upvotes: 2

Related Questions