CODEWITHSUNDEEP
jsonelasticsearchlogstash
Rachel
Rachel

Reputation: 727

Filtering JSON/non-JSON entries in Logstash

I have a question about filtering entries in Logstash. I have two different logs coming into Logstash. One log is just a std format with a timestamp and message, but the other comes in as JSON.

I use an if statement to test for a certain host and if that host is present, then I use the JSON filter to apply to the message... the problem is that when it encounters the non-JSON stdout message it can't parse it and throws exceptions.

Does anyone know how to test to see if an entry is JSON coming in apply the filter and if not, just ignore it?

thanks

  if [agent][hostname] == "some host"
  # if an entry is not in json format how to ignore?
  {
    json {
      source => "message"
      target => "gpfs"
     }
  }

Upvotes: 1

Views: 747

Answers (1)

Luminance
Luminance

Reputation: 918

You can try with a grok filter as a first step.

grok {
    match => {
      "message" => [
        "{%{GREEDYDATA:json_message}}",
        "%{GREEDYDATA:std_out}"
     ]
  }
}
if [json_message]
{
    mutate {
        replace => { "json_message" => "{%{json_message}}"}
    }
    json {
        source => "json_message"
        target => "gpfs"
     }
}

Probably there is a more cleaner solution then this, but it will do the job.

Upvotes: 1

Related Questions