CODEWITHSUNDEEP
tomcatlogstashlogstash-groklogstash-configurationelastic-stack
mathakoot
mathakoot

Reputation: 1742

Grok pattern for tomcat logs gives compile error on Grok Debugger

I am trying to setup ELK for a Java application. The tomcat logs are produced using log4j. To write a test a pattern, I am using Grok Debugger. But on the debugger it always shows

Compile ERROR

My log sample:

YYYY-MM-DD HH:MM:SS,SSS INFO : [so-me-uni-que-id] com.xx.xx.xx.xx.xx - log message here

My grok filter:

filter {   if [type] == "tomcat" {     grok {       match => { "message" => "%{TOMCATLOG}" }     }     date {       match => [ "timestamp", "yyyy-MM-dd HH:mm:ss,SSS" ]     } } }

My pattern:

TOMCATLOG %{TOMCAT_DATESTAMP:timestamp} \| %{LOGLEVEL:level} \| %{UNIQUEID:uniqueid}\| %{JAVACLASS:class} - %{JAVALOGMESSAGE:logmessage}

Upvotes: 1

Views: 3586

Answers (2)

Anton
Anton

Reputation: 730

I also had problems with Tomcat. Also don't need to forget that %LOGLEVEL pattern doesn't contains all levels for Tomcat (CONFIG, FINE, FINER, FINEST). It could be

TOMCAT_LOGLEVEL ([A-a]lert|ALERT|[T|t]race|TRACE|[D|d]ebug|DEBUG|[N|n]otice|NOTICE|[I|i]nfo|INFO|[W|w]arn?(?:ing)?|WARN?(?:ING)?|[E|e]rr?(?:or)?|ERR?(?:OR)?|[C|c]rit?(?:ical)?|CRIT?(?:ICAL)?|[F|f]atal|FATAL|[S|s]evere|SEVERE|EMERG(?:ENCY)?|[Ee]merg(?:ency)?|CONFIG|FINE|FINER|FINEST)

I added all grok patterns in one place (Nginx, Tomcat, Spring): https://gist.github.com/petrov9/4740c61459a5dcedcef2f27c7c2900fd

Hope it will save your time

Upvotes: 0

Alain Collins
Alain Collins

Reputation: 16362

The basic issue is that your pattern doesn't match your input. Look at the beginning:

YYYY-MM-DD HH:MM:SS,SSS INFO : [so-me-uni-que-id]

%{TOMCAT_DATESTAMP:timestamp} \| %{LOGLEVEL:level} \| %{UNIQUEID:uniqueid}\|

Your pattern has escaped pipes ("|"), but the input doesn't use them.

I also don't see that TOMCAT_DATESTAMP is in the default patterns, but maybe it's buried somewhere.

Start at the left side, matching one piece at a time in the debugger.

%{TIMESTAMP_ISO8601} %{WORD:level} : \[%{GREEDYDATA:uniqueid}\]

Then keep working your way across, grabbing more stuff into your pattern. Note that literals (":" and the escaped "[") become part of your pattern.

Good luck!

Upvotes: 1

Related Questions