CODEWITHSUNDEEP
outputlogstash
Fred Mériot
Fred Mériot

Reputation: 4357

Logstash : how to output a selection of fields

I have a logstash exe that read JSON events from a RabbitMQ queue. 

input {
  rabbitmq  {
    codec => json
    ...
  }
}

I need to have 2 outputs. The first one is a MongoDB output with the entire JSON document (no problem, it works), and the second is another rabbitMQ queue but I don't need the entire JSON. I just need a selection of fields.

How can I do that?

Thank you

Upvotes: 1

Views: 4329

Answers (2)

coffeepac
coffeepac

Reputation: 634

Or you can use the following codec: https://github.com/payscale/logstash-codec-fieldselect It selects some fields for output. Just like the above but without a ruby filter.

Upvotes: 0

Alcanzar
Alcanzar

Reputation: 17155

You'd use a mutate filter with a remove_field entry to remove all of the fields you don't want. If you don't know what all of the fields are that you need to remove, you'll need to create a ruby filter the iterates over the event and removes anything that isn't in your desired list.

That ruby code would look something like this:

filter {
    ruby {
        code => " 
                event.to_hash.each {|k,v|
                        if (!['a','b','c'].include?(k))
                                event.remove(k)
                        end
                }"
    }
}

where ['a','b','c'] is the list of fields you want to keep.

Upvotes: 3

Related Questions