CODEWITHSUNDEEP
prometheusgrafanagrafana-lokipromtail
solarflare
solarflare

Reputation: 1092

Using Promtail to sum log line values - Pipeline Stages - Metrics

Given a log lines such as:

[Some info] myproject - number of values: 50
[Some info] myproject - number of values: 100
[Some info] myproject - number of values: 20

I am looking to add these values up and display them in Grafana. i.e. given a 10m interval, Grafana will display 'myproject' had a total of 170 values.

I've been looking at https://grafana.com/docs/loki/latest/clients/promtail/stages/metrics/#gauge which seems to do just that, and here is my config within Promtail:

server:
  http_listen_port: 9080

positions:
  filename: /tmp/positions.yaml

clients:
  - url: http://<host>:3100/loki/api/v1/push

scrape_configs:
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: nifi_server_1
      __path__: /var/log/nifi*
  pipeline_stages:
  - regex:
      expression: 'number of values: (?P<numValues>[0-9]{1,4})'
  - labels:
      numOfvalues: numValues
  - metrics:
      number_of_values:
        type: Gauge
        description: "sum of number of values"
        source: numValues
        config:
          action: add

However I end up with strange results where Grafana things each individual 'value' is a label in itself, rather than adding them up: enter image description here

Upvotes: 1

Views: 2260

Answers (1)

Florin Marin
Florin Marin

Reputation: 410

The metrics pipeline works by adding all your existing labels to the metrics as well.

Because you are adding the labels in the labels pipeline, they will also appear for the metrics.

You have to remove this from your stages. If you use promtail with --inspect, you will see those labels added.

  - labels:
      numOfvalues: numValues

Upvotes: 0

Related Questions