CODEWITHSUNDEEP
logstashlogstash-groklogstash-configuration
User588233
User588233

Reputation: 491

Filtering messages out of logstash.conf

My logstash.conf can be seen below.

How would I go about filtering out messages which include a specific string? In this case, some of my messages are reading as "DEBUG: xxx-xxx-xxx", and I would like for these to be filtered OUT of logstash.

input {
  tcp {
    port => 5000
    type => syslog
  }
  udp {
    port => 5000
    type => syslog
  }
}
filter {
  if [loglevel] == "debug" {
      drop { }
  }

  if [type] == "syslog" {
    grok {
      match => { 
      "message" => "%{SYSLOG5424PRI}%{NONNEGINT:ver} +(?:%  {TIMESTAMP_ISO8601:ts}|-) +(?:%{HOSTNAME:containerid}|-) +(?:%{NOTSPACE:containername}|-) 
      +(?:%{NOTSPACE:proc}|-) +(?:%{WORD:msgid}|-) +(?:%{SYSLOG5424SD:sd}|-|) +%{GREEDYDATA:msg}" }
}   
  syslog_pri { } 
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }   
   if !("_grokparsefailure" in [tags]) {
     mutate {
        replace => [ "@source_host", "%{syslog_hostname}" ]
        replace => [ "@message", "%{syslog_message}" ]
      }   
    }   
    mutate {
      remove_field => [ "syslog_hostname", "syslog_message", "syslog_timestamp" ]
    }   
  }

}

output {
  elasticsearch { host => "elasticsearch" }
  stdout { codec => rubydebug }
}

EDIT: I should clarify that it is the GREEDYDATA:msg field that I wish to drop if it includes a "DEBUG" message.

Upvotes: 1

Views: 1482

Answers (1)

Alain Collins
Alain Collins

Reputation: 16362

You're looking for drop:

filter {
    if [myField] == "badValue" {
        drop { }
    }
}

While it's better to use exact matches, you can also do regexp conditionals:

filter {
    if [myField] =~ "DEBUG" {
        drop { }
    }
}

Upvotes: 1

Related Questions