CODEWITHSUNDEEP
fluentd
Robert Strauch
Robert Strauch

Reputation: 12906

Apply fluentd configuration depending on environment variable

Given the following example configuration for fluentd, how could I enable or disable the configuration for Elasticsearch based on an environment variable ENABLE_ELASTICSEARCH?

fluentd will be started in a Docker container, so maybe a custom script could help? Or are there any mechanisms built into fluentd which I could use? The documentation mentions "Embedding Ruby Code" but this seems to work only for configuration values?

<match mytag>
    @type copy
    copy_mode deep
    <store>
        @type relabel
        @label @OPENSEARCH 
    </store>
    <store>
        @type relabel
        @label @ELASTICSEARCH 
    </store>
</match>

<label @OPENSEARCH>
   @include outputs/fluent-opensearch.conf 
</label>

<label @ELASTICSEARCH>
   @include outputs/fluent-elasticsearch.conf 
</label>

Upvotes: 0

Views: 117

Answers (1)

Imran
Imran

Reputation: 6265

Below is how I would address it but there can be more better and efficient ways. Let me know your thoughts!!.

Create another fluent-null.conf which will discard events with @type null match.

<match>
  @type null
</match>

In your fluent.conf, you can update @LABEL sections to use environment variables for @include file names.

<label @OPENSEARCH>
   @include "#{ENV['FLUENTD_OPENSEARCH']}"
</label>

<label @ELASTICSEARCH>
   @include "#{ENV['FLUENTD_ELASTICSEARCH']}"
</label>

Now, you can either point environment variables to actual conf file or null one.

docker run --rm -e FLUENTD_ELASTICSEARCH="fluent-elasticsearch.conf" -e FLUENTD_OPENSEARCH="fluent-opensearch.conf" localfluentd

OR with below command ELASTICSEARCH will be discarded.

docker run --rm -e FLUENTD_ELASTICSEARCH="fluent-null.conf" -e FLUENTD_OPENSEARCH="fluent-opensearch.conf" localfluentd

More explanation in below blog which gives sample repo to test this approach.

https://ranbook.cloud/answers/fluentd-conditional-matches/

Upvotes: 0

Related Questions