CODEWITHSUNDEEP
jsontimestamplogstash
rkpt
rkpt

Reputation: 39

Logstash use JSON field date instead of @timestamp

I'm facing this issue. I'm trying to use custom JSON log date as my "usable" date instead of the @timestamp date field.

My JSON file to be processed by Logstash (comming from filebeat):

{
    "start":    {
       "timestamp": {
            "time": "Wed, 04 Apr 2018 09:36:39 GMT",
            "timesecs": 1522834599
        }
    }
}

My logstash.yml file :

input {
  beats {
    port => 1337
    codec => "json_lines"
  }
}

filter {
   date {
      match => [ "time", "EEE, dd MM yyyy hh:mm:ss ZZZ" ]
   }
}

output {
  stdout {
    codec => rubydebug
  }
  elasticsearch {
    hosts => "localhost:9200"
    index => "testing"
  }
}

Also tried to :

match => [ "[start][timestamp][time]", "EEE, dd MM yyyy hh:mm:ss ZZZ" ]

Still no luck.

Any help would be welcome.

Cheers,

Upvotes: 2

Views: 1161

Answers (2)

Sufiyan Ghori
Sufiyan Ghori

Reputation: 18763

When you match the date using date filter, it stores the matching timestamp into the given target field. If target not provided, it will simply update the @timestamp field of the event with the new matching time.

target

Store the matching timestamp into the given target field. If not provided, default to updating the @timestamp field of the event.

target will automatically create a field if it doesn't exists.

So following code is enough,

date {
     match => [ "[start][timestamp][time]", "EEE, dd MMM yyyy HH:mm:ss z" ]
     target => "newTimeField"
     locale => "en"
     remove_field => [ "[start][timestamp][time]" ]  
}

remove_field above is another common option available for date filter. It is used to delete [start][timestamp][time] field once its stored in a new field.

Upvotes: 1

rkpt
rkpt

Reputation: 39

Got to the solution like this :

mutate {
     add_field => {
             "mytime" => ""
     }
}

date {
     match => [ "[start][timestamp][time]", "EEE, dd MMM yyyy HH:mm:ss z" ]
     target => "mytime"
     locale => "en"
}

Upvotes: 0

Related Questions