CODEWITHSUNDEEP
elasticsearchlogstashlogstash-grok
M.Ramzan
M.Ramzan

Reputation: 317

failed to filter logs with grok

I want to filter the log I have with grok and I have gone through many documents but I couldn't find a solution for my problem. I have a log like this,

2017-01-24|15:00:11|UpToDate.Editorial.Service.Topic.IcgDataIslandSaveUtil|1|[(null)]| - Step 0, Start saving data island. 0.0010006 seconds since last step, 0.0020005 seconds since start.

which is in the format of,

date|time|class|level|user|log

I have created a conf file for this as below,

input {
    beats {
        port => "5044"
    }
}
filter {
  grok {
    match => { "message" => "%{YEAR}-%{MONTHNUM}-%{MONTHDAY}|%{HOUR}:%{MINUTE}:%{SECOND}|%{GREEDYDATA:class}|%{NUMBER:level}|%{GREEDYDATA:user}|%{GREEDYDATA:log}\n" }
  }
}
output {
    elasticsearch {
        hosts => [ "localhost:9200" ]
    }
}

But it failed to filter out those details and it just index the whole log. Can anyone point out what I am doing wrong here? Any help would be much appreciated.

Upvotes: 0

Views: 34

Answers (1)

MrSimple
MrSimple

Reputation: 599

You had special characters in the pattern. Use this:

%{YEAR}-%{MONTHNUM}-%{MONTHDAY}\|%{HOUR}:%{MINUTE}:%{SECOND}\|%{GREEDYDATA:class}\|%{NUMBER:level}\|%{GREEDYDATA:user}\|%{GREEDYDATA:log}<br/>

I removed the '\n' at the end, it messed up the parsing for me. And if you run in grok pattern problems again, use this site, it helps a lot:
http://grokconstructor.appspot.com/do/match

Upvotes: 1

Related Questions