CODEWITHSUNDEEP
elasticsearchlogstash-grok
Dmitry R
Dmitry R

Reputation: 3176

Logstash Grok Modifying and overwriting @timestamp

My logs having timestamps in format of: Nov 19 00:06:37

I need to have this format to be converted to ISO 8601 timestamp and used as @timestamp field?

What would be the correct configuration to handle this? I have the following config right now:

filter {
  grok {
    match => { "message" => "%{SYSLOGTIMESTAMP:logTimestamp} %{USERNAME:myServer} %{USERNAME:myProcessName}: %{INT:operationType} %{WORD}, \"%{INT} %{WORD}, %{WORD} %{WORD}: /%{WORD}/%{WORD:clientId}/%{WORD}, %{WORD} %{WORD}: %{WORD:myId1}, \"%{WORD:status}\", %{WORD}-%{WORD}: %{INT:sessionId}" 
    }
  }
  date {
    match => [ "logTimestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
  }
}

Upvotes: 1

Views: 4074

Answers (1)

Sahishnu Patil
Sahishnu Patil

Reputation: 119

To update the value of @timestamp generated by Logstash as a metadata parameter, to the value timestamp or logTimestamp generated by grok, use date plugin to match and update the value of @timestamp.

    grok {
      match => { "message" => "%{COMBINEDAPACHELOG}" }  # --> COMBINEDAPACHELOG gives timestamp as output
    }
    date {
      match => [ "timestamp", "dd/MMM/YYYY:HH:mm:ss Z"]  #  --> Here match the pattern of timestamp to the format
      target => "@timestamp"    # --> Here the @timestamp value will be updated
      remove_field => ["timestamp"]
    }
    useragent {
      source => "agent"
    }
  }

Reference:
date plugin - https://www.elastic.co/guide/en/logstash/current/plugins-filters-date.html#plugins-filters-date-target

Upvotes: 4

Related Questions