CODEWITHSUNDEEP
elasticsearchlogstashgroklogstash-groklogstash-forwarder
Sharad Ahire
Sharad Ahire

Reputation: 788

Update @timetamp field in logstash with custom timestamp value

I have following logstash config file for parsing following exception stack trace.

stacktrace

2015-03-02 09:01:51,040 [com.test.MyClass] ERROR - execution resulted in Exception
com.test.core.MyException
    <exception line1>
    <exception line2>
2015-03-02 09:01:51,040 [com.test.MyClass] ERROR - Encountered Exception, terminating execution

Config File:

input {
stdin {}
}

filter {
  multiline {
      pattern => "(^%{TIMESTAMP_ISO8601}) | (^.+Exception+) | (^.+Error+)"
      negate => true
      what => "previous"
    }
}

output {
stdout { codec => rubydebug }
}

I am able to parse stack trace into single logstash field name message. However i want update the @timestamp with timestamp of first exception line i.e. 2015-03-02 09:01:51,040

Currently it has been taking default timestamp for @timestamp

Any help would appreciated.

Upvotes: 1

Views: 2738

Answers (1)

Ban-Chuan Lim
Ban-Chuan Lim

Reputation: 7890

You need to use GROK filter to extract the time value and then use DATE filter parse the value into @timestamp

For example:

input {
        stdin {
                codec => multiline {
                        pattern => "(^%{TIMESTAMP_ISO8601}) | (^.+Exception+) | (^.+Error+)"
                        negate => true
                        what => "previous"
                }
        }
}

filter {

        grok {
                match => ["message" , "%{TIMESTAMP_ISO8601:logtime} %{GREEDYDATA:msg}"]
        }

        date {
                match => ["logtime", "YYYY-MM-dd HH:mm:ss,SSS"]
        }
}

output {
    stdout { codec => rubydebug }
}

Beside, use the multiline in the input instead of in filter is, the multiline in filter will collapse the message into an message array, instead of a single mesage string. So, it will cause grok and date filter failed.

Upvotes: 3

Related Questions