CODEWITHSUNDEEP
elasticsearchkibanafluent-bit
neilH
neilH

Reputation: 3438

fluentbit - exclude specific kubernetes annotation from exported log metadata

I have some pods that contain an annotation with a very long value. I feel this is causing issues when fluentbit attempts to export the logs to elasticsearch. I don't need this particular annotation to be exported with the log metadata. I've tried to prevent this annotation from being exported by turning off annotations in the fluentbit kubernetses filter like so:

        [FILTER]
            Name                kubernetes
            Match               kube.*
            Merge_Log           On
            Keep_Log            Off
            K8S-Logging.Parser  On
            K8S-Logging.Exclude On
            Annotations         Off

However, this causes us other issues, as we need to some of the other annotations the filter is removing. So my question is, is there a way to exclude a single/specific annotation from the metadata, rather than exclude all the annotations?

Upvotes: 0

Views: 2317

Answers (1)

Kin Cheung
Kin Cheung

Reputation: 860

Use these filters:

  1. lift "kubernetes" metadata one level up
  2. remove the data that you don't need
  3. put the remaining data back into "kubernetes"
[FILTER]
    Name                nest
    Match               application.*
    Operation           lift
    Nested_under        kubernetes
    Add_prefix          Kube.

[FILTER]
    Name                modify
    Match               application.*
    Remove              Kube.annotation.<Metadata_1>
    Remove              Kube.annotation.<Metadata_2>
    Remove              Kube.annotation.<Metadata_3>

[FILTER]
    Name                nest
    Match               application.*
    Operation           nest
    Wildcard            Kube.*
    Nested_under        kubernetes
    Remove_prefix       Kube.

Upvotes: 0

Related Questions