CODEWITHSUNDEEP
elasticsearchlogstashkibana
Mahith Geallapally
Mahith Geallapally

Reputation: 21

Use non whitelisted fields in Logstash if statement

I have a logstash configuration where I used prune to whitelist few fields.

prune {
   whitelist_names => ["id","name"]
  }

The problem is, I need to use an if condition in the output on a field other than id field, eg: "type". But since I have not whitelisted "type", the if condition is not working.

 if ( [type] in  ["abc","efg"] )  {
        elasticsearch {
            action => "update"
            hosts => [ "localhost:9200" ]
            index => "index"
            document_id => "%{id}"
            doc_as_upsert => true
        }
      }

How can I use non whitelisted field in if condition?

Upvotes: 0

Views: 250

Answers (1)

glenacota
glenacota

Reputation: 2547

Before your prune filter, add a mutate filter to copy the value of the field you're going to delete (type) into a new metadata field. Then, prune. Then, use the new metadata field in your output condition.

...
filter {
  ...
  mutate {
    add_field => {
      "[@metadata][type]" => "%{type}"
    }
  } 
  prune {
   whitelist_names => ["id","name"]
  }
  ...
}

output {
  if [@metadata][type] in  ["abc","efg"] {
    elasticsearch {
      action => "update"
      hosts => [ "localhost:9200" ]
      index => "index"
      document_id => "%{id}"
      doc_as_upsert => true
    }
  }
}

Upvotes: 2

Related Questions