CODEWITHSUNDEEP
prometheustraefikkube-prometheus-stack
Martijn Heemels
Martijn Heemels

Reputation: 3659

Replace a character in a Prometheus label's value

I have a label crawled by Prometheus that contains hostnames but uses dashes as separators instead of dots. For example domain=www-example-com.

I'm trying to replace that with a label containing dot-separated values, e.g. turn it into domain=www.example.com but can't find a way to do that. In other words a character replace ('-' to '.'), regsubst or something like that. Using regex capture groups isn't working since the hostname has a variable number of dashes.

I'm using kube-prometheus' metricRelabelings in the ServiceMonitor that crawls Traefik's metrics.

Is there really no way to do simple string substitution? This GitHub ticket from 2018 claims it isn't possible and wouldn't be added to Prometheus either.

Did I miss anything?

Upvotes: 2

Views: 6619

Answers (1)

valyala
valyala

Reputation: 17860

Prometheus doesn't provide relabeling actions for replacing some chars in label values or label names. But such relabeling actions are provided by an alternative Prometheus-like monitoring solution I work on - VictoriaMetrics:

  • action: replace_all replaces all the regex occurrences in source_labels separated by separator with the replacement and stores the result to target_label. For example, the following relabeling rule replaces all the - chars with . in hostname label name:
- action: replace_all
  source_labels: [hostname]
  target_label: hostname
  regex: "-"
  replacement: "."
  • action: labelmap_all replaces all the regex occurrences in all the label names with the replacement. For example, the following relabeling rule replacess all the - chars in label names with .:
- action: labelmap_all
  regex: "-"
  replacement: "."

See more details on these relabeling actions in these docs.

VictoriaMetrics also provides the function label_transform(), which can be used in queries for substituting the specified regexp patterns in label values with the given replacements. For example, the following query substitutes all the - chars with . chars in hostname label of the some_metric metric:

label_transform(some_metric, "hostname", "-", ".")

Upvotes: 4

Related Questions