CODEWITHSUNDEEP
elasticsearchlogstashfilebeat
Rohit
Rohit

Reputation: 75

Can grok expression be written to enrich log files in FileBeat before sending to Logstash / elastic search

My use case is to ship log files from various applications to Elasticsearch so that I can view them from Kibana.

I wanted to know can Filebeat be configured for grok expression so that application team can manage their log parsing at their end and central logging system / deployment is unaffected? If it can be then the need for Logstash is questionable. I can see Filebeat supports multi-line expressions, but can I define other grok expression and add/remove fields by parsing the message?

Upvotes: 2

Views: 583

Answers (1)

A J
A J

Reputation: 2583

Filebeat does not perform grok processing. So you will need to send the data to Logstash for processing.

There is a new feature coming in Elasticsearch 5.0 called Ingest Node that will allow some simple grok processing to be performed without needing Logstash (i.e. Filebeat -> Elasticsearch).

Upvotes: 1

Related Questions