CODEWITHSUNDEEP
filterlogstashfilteringusinglogstash-grok
ElasticSearchUser
ElasticSearchUser

Reputation: 361

Logstash filtering using grok to filter the log with []

I have the below log captured in elastic serach which we view via kibana gui. Now we need to break down the log into datetime stamp, log level, object name, description etc to display them in a dashboard.

{"message":" Mon Dec 15 2014 05:55:11 [test][crypto][info] mpgw(Test): tid(135196353)[request][10.259.268.129]: certificate validation succeeded for '/C=US/O=Company/OU=Web Servers/CN=abcqa-client.company.com' against 'CompanyCA_ValCred' \r","@version":"1","@timestamp":"2014-12-15T10:55:19.168Z","host":"testserver","path":"/opt/store/device_logging/QA1_filter2.log.20141215055512"}

I am using grok debugger (https://grokdebug.herokuapp.com/) to test before updating logstash conf file. I have problem with [],:, and (). Without these, i could get the output like below.

Input: Mon Dec 15 2014 05:55:11 test crypto INFO mpgwTest tid135196353 request 10.255.215.129 certificate validation succeeded for '/C=US/O=Company/OU=Web Servers/CN=abcqa-client.ompany.com' against 'CompanyCA_ValCred'

Pattern: %{WORD:day} %{MONTH:month} %{MONTHDAY:monthday} %{YEAR:year} %{TIME:time} (?:%{WORD:DOMAIN}) (?:%{WORD:Objecttype}) (?:%{LOGLEVEL:level}) (?:%{WORD:DPObject}) (?:%{WORD:tid}) (?:%{WORD:flowtype}) (?:%{IP:ClientIP}) (?:%{GREEDYDATA:Description})
I am getting output properly.

How can I omit [],:,().?

Upvotes: 0

Views: 844

Answers (1)

user4383461
user4383461

Reputation:

You can escape special regex characters by putting a backslash ('\') in front of them.

For example the first part of your pattern can be written:

%{WORD:day} %{MONTH:month} %{MONTHDAY:monthday} %{YEAR:year} %{TIME:time} \[(?:%{WORD:DOMAIN})\]\[(?:%{WORD:Objecttype})\]\[(?:%{LOGLEVEL:level})\]

Upvotes: 3

Related Questions