CODEWITHSUNDEEP
logstash
Raymond Nakampe
Raymond Nakampe

Reputation: 371

Filtering specific lines

I am currently trying to filter specific lines from my log file. My lines in the log file are of the following pattern.

[8/05/13 14:24:55.468] RuntimeErrorI E LaError
[8/05/13 14:24:55.468] AbcdEfg W SomeWarning

where the first is the date, time, application name and the Log level( WARNING, ERROR, TRACE etc) followed by the error message or warning message or any other messages.

So what I am trying to get is the log level errors only and not other log levels. I have the following which I am playing around with but I am not getting console output at all. I think I am making a mistake somewhere in my grep for checking if its E(Error)

input {

  file {
  type => "database"
  path => "/home/nakampe/Desktop/file.log"
  }

}

filter {

 grok {

    pattern =>  "[%{MONTHDAY:date} / %{MONTH:month} / %{YEAR:year}  %{TIME:time}] %WORD:application} %{WORD:levelType}  %{WORD:message}"

  }


#Here I want to only consider log levels of E (ERROR) and not others
  grep{

  match => ["levelType", "E"]

  }

}

output {

   elasticsearch { 

   embedded => true 
   }

stdout { 
  message => "%{@message}"
  }     
}

Upvotes: 3

Views: 5806

Answers (3)

user3714588
user3714588

Reputation: 1

under the grok section

if [levelType] != "ERROR" {
        drop { }
}

Upvotes: 0

Raymond Nakampe
Raymond Nakampe

Reputation: 371

 grok { 

pattern =>  "%{MONTHDAY:date}/%{MONTHNUM:month}/%{YEAR:year} %{TIME:time} %{WORD:sast}]      

    %{INT:threadId} %{WORD:applicationName}%{SPACE:space}%{WORD:logLevel} %     {WORD:errorMessage}"

}

My mistake was with the pattern, after debugging using http://grokdebug.herokuapp.com/ all was ok. A very useful tool.

Upvotes: 1

jayeff
jayeff

Reputation: 148

I think you need to chain both filters otherwise they will be applied independently to the original line. You can do this by adding a tag in the grok filter and react to this in the grep filter:

grok {
  pattern => ...
  add_tag => [ "groked_input" ]
}

and 

grep {
  match => ...
  tags => [ "groked_input"]
}

Hope that helps! :)

Upvotes: 0

Related Questions