CODEWITHSUNDEEP
logstash
Sergey Romanovsky
Sergey Romanovsky

Reputation: 4574

logstash fails to match a grok filter

I'm stuck. I cannot get why grok fails to match a simple regex under logstash. grok works just fine as a standalone thing. The only pattern which works for me is ".*" everything else just fails.

$ cat ./sample2-logstash.conf
input {
    stdin {}
}

filter {
    grok {
        match => [ "message1", "foo.*" ]
        add_tag => [ "this_is_foo" ]
        tag_on_failure => [ "STUPID_LOGSTASH" ]
    }
}

output {
    stdout { codec => json_lines }
}

Here's the output:

$ echo "foo" |~/bin/logstash-1.4.0/bin/logstash -f ./sample2-logstash.conf 
{"message":"foo","@version":"1","@timestamp":"2014-05-07T00:32:49.915Z","host":"serega-sv","tags":["STUPID_LOGSTASH"]}

Looks like I missed to do something in logstash because vanilla grok works just fine:

$ cat grok.conf
program {
    file "./sample.log"
    match {
        pattern: "foo.*"
        reaction: "LINE MATCHED! %{@LINE}"
    }
}

Plain grok's output:

$ echo "foo" > ./sample.log; grok -f grok.conf 
LINE MATCHED! foo

Thanks!

Upvotes: 1

Views: 1553

Answers (2)

Dan Garthwaite
Dan Garthwaite

Reputation: 3526

Everything @Ben Lim said. The very next section of the documentation shows how to apply semantics to generic regex syntax:

filter {
  grok {
    match => [ "message", 
      "^(?<ip>\S+) (?<verb>\S+) (?<request>\S+) (?<bytes>\S+) (?<delay>\S+)$"
    ] 
  }
}

Upvotes: 1

Ban-Chuan Lim
Ban-Chuan Lim

Reputation: 7890

You configuration have error. The grok match field is message, instead of message1.

Then, at logstash grok page there is an example to show how to use grok. I think you have misunderstand. For example, if your log is 

55.3.244.1 GET /index.html 15824 0.043

The grok pattern for logstash is 

%{IP:client} %{WORD:method} %{URIPATHPARAM:request} %{NUMBER:bytes} %{NUMBER:duration}

For %{IP:client}, The first parameter (IP) is grok pattern, the second parameter(client) is the field you want to put this message.

Upvotes: 3

Related Questions