CODEWITHSUNDEEP
logstashlogstash-grok
ennon
ennon

Reputation: 129

grok filter logstash JSON parse error, original data now in message field

I'm using logstash with a configuration input{rabbitmq} filter{grok} output{elastic}

From rabbit I receive nginx logs in this format :

- - [06/Mar/2017:15:45:53 +0000] "GET /check HTTP/1.1" 200 0 "-" "ELB-HealthChecker/2.0"

and I'm using grok filter as simple as follow :

filter{
    if [type] == "nginx" {
        grok{
            match => { "message" => "%{NGINXACCESS}" }
        }   
    }
}

and the pattern is 

NGUSERNAME [a-zA-Z\.\@\-\+_%]+
NGUSER %{NGUSERNAME}
NGINXACCESS %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:timestamp}\] "%{WORD:verb} %{URIPATHPARAM:request} HTTP/%{NUMBER:httpversion}" %{NUMBER:response} (?:%{NUMBER:bytes}|-) (?:"(?:%{URI:referrer}|-)"|%{QS:referrer}) %{QS:agent}

I tried the pattern in grok debugger and it seems to work just fine but running the pipeline i get this error

[2017-03-06T16:46:40,692][ERROR][logstash.codecs.json     ] JSON parse error, original data now in message field {:error=>#, :data=>"- - [06/Mar/2017:16:46:40 +0000] \"GET /check HTTP/1.1\" 200 0 \"-\" \"ELB-HealthChecker/2.0\""}

it seems like someone(logstash?) is adding \ to the result...

hope to get some help, thanks!

Upvotes: 1

Views: 5265

Answers (1)

Will Barnwell
Will Barnwell

Reputation: 4089

This does not seem to be a grok error at all. if grok fails to parse it will add a tag _grokparsefailure to your event. A JSON parse error would be due to your input trying to read codec => json {} when your log format is plainly not JSON. Make sure that your input plugin that is handling these log types is using codec => plain or an appropriate type.

See logstash codecs for more info.

Upvotes: 1

Related Questions