CODEWITHSUNDEEP
logstash-grok
Eva
Eva

Reputation: 11

Logstash grok pattern for kafka logs

I'm trying to ship Kafka logs to elk for which i'm using below grok filter in logstash:

grok {
match => {"actual_message" => "%{SYSLOG5424SD} %{LOGLEVEL} %{GREEDYDATA}"}
}

Kafka Log format:

[2019-09-24 12:54:11,546] DEBUG [Controller id=0] Topics not in preferred replica Map() (kafka.controller.KafkaController)
[2019-09-24 12:54:11,546] TRACE [Controller id=0] Leader imbalance ratio for broker 0 is 0.0 (kafka.controller.KafkaController)

When I use this, the 'timestamp' of actual logs in Kibana is not shown correctly. To clarify, if you notice the 'message' section in below image, the event occurred on 24th September but the "@timestamp" shows 26th September, the date and time when I started shipping logs to elk. Can anyone tell me the actual grok pattern which would fix the issueLogs in Kibana

Upvotes: 1

Views: 1208

Answers (1)

Ishara Dayarathna
Ishara Dayarathna

Reputation: 3611

Try this:

\[%{TIMESTAMP_ISO8601:timestamp}] %{LOGLEVEL} %{GREEDYDATA}

Upvotes: 0

Related Questions