CODEWITHSUNDEEP
logstash
Orkun
Orkun

Reputation: 512

How to use mutate filter plugin in output

I have a logstash configuration. With that configuration logstash do some operation in filter and send outputs. Everything works well.

I want one more elasticsearch output in same configuration file. I mean after parsing the logs, logstash send results to one index after that removing some fields and send them to another index. My aim is in below example.

output {
 elasticsearch {
      ..index1..
 }
 mutate {
   remove_field ...
} 
mutate {
   add_field ..
}
elasticsearch {
  ... index2 ...
}

}

But i can not work with mutate plugin in output ? How can i achieve this ?

Thanks for answering

Upvotes: 1

Views: 2514

Answers (1)

whites11
whites11

Reputation: 13260

mutate is a filter plugin, so it will only work inside the filter block.

What you can do maybe is use the clone filter to duplicate your event, apply different filters to the original and cloned event and then deliver the two copies to different outputs accordingly.

It could look roughly like this

filter {
  clone {
    clones => ["cloned"]
  }
  # apply here filters that should be applied to both the primary and secondary event
  if [type] == "cloned" {
    # apply here filters that should be applied to the secondary event
  }
}

output {
  if [type] == "cloned" {
    elasticsearch {
      ... index2 ...
    }
  } else {
    elasticsearch {
      ... index1 ...
    }
  }
}

See https://discuss.elastic.co/t/can-i-filter-the-output-data-separately/36064/5

Upvotes: 2

Related Questions