CODEWITHSUNDEEP
filtersplitlogstash
Labynocle
Labynocle

Reputation: 4612

Logstash - using mutate with split

I'm trying to use the mutate filter with the split method, to extract a part of a field retrieved by the json filter. But I didn't find how to exploit the result of the split. Here's my logical configuration (only the simple json part works well):

filter {
        if [type] == "phperrors" {
                json {
                        source => "message"
                        target => "phperror_log"
                        remove_field => "message"
                        add_tag => [ "php", "error" ]
                }
                mutate {
                        split => ["[phperror_log][server]", "."]
                        #if [phperror_log][server][1] {
                                add_field => [ "pop", "%{[phperror_log][server][1]}" ]
                        #}
                }
        }
}

As I said the json part works well and the phperror_log.server field exists (and the value is something like node01.ny or node01.par or node02).

But all I want is to create a new field if the split create an array of 2 fields and I suppose I'm not doing the right thing with my [phperror_log][server][1].

Have you any clues, advices or best practices to resolve this kind of use case?

Thanks in advance

edit: I edit with a conf that should work with the split... however the if is not ok

last edit with a nice conf according to the tips and advices from Alain:

filter {
        if [type] == "phperrors" {
                json {
                        source => "message"
                        target => "phperror_log"
                        remove_field => "message"
                        add_tag => [ "php", "error" ]
                }
                if [phperror_log][server] =~ /\./ {
                        mutate {
                                split => ["[phperror_log][server]", "."]
                                add_field => [ "pop", "%{[phperror_log][server][1]}" ]
                                add_field => [ "errorhost", "%{[phperror_log][server][0]}" ]
                        }
                } else {
                        mutate {
                                add_field => [ "pop", "not defined" ]
                                add_field => [ "errorhost", "%{[phperror_log][server]}" ]
                        }
                }
                mutate {
                        remove_field => [ "[phperror_log][server]" ]
                }
        }
}

Upvotes: 4

Views: 17088

Answers (1)

Alain Collins
Alain Collins

Reputation: 16362

Once you split "node01.ny", your field will be ["node01", "ny" ]. Your example doesn't really show what you want to do with these split values, but the basic one would be to see if a value is contained in the array:

if "ny" IN [phperror.server] {
}

Note that add_field only runs if the filter is successful. As such, you shouldn't need to check the array before adding the field.

That said, the split will only work if the field contained a ".". You could conditional on that:

if phperror.server =~ /\./ {
  mutate {
     add_field => [ "hey", "i saw a period" ] 
  }
}

You example might show that you want to remove part of the value of the original field. Try gsub:

gsub => [ "phperror.server", "\..*", "" ]

This would remove everything after the ".".

Hope that helps.

Upvotes: 2

Related Questions