CODEWITHSUNDEEP
logstashlogstash-grok
Kamikaze K
Kamikaze K

Reputation: 191

Grok pattern from line - custom date

I am trying to use a grok pattern to get the fields from the following. any help with this especially with the custom date?

0E7 10JUN21 23:37:53.8 Track 1 EZ: 100 EX: 72 Speed: 36 mph

I need to get the fields;

"date" "time" "Track 1" "EZ" "EX" "Speed"

I have tried to get the date pattern but with no luck

Upvotes: 1

Views: 2370

Answers (1)

leandrojmp
leandrojmp

Reputation: 7463

You will need a custom grok to get the date, month and year in separated fields, then you will need to capitalize the month field and after that add a new field with the complete date string to use in the date filter.

To parse your date 10JUN21 into separated fields you can use the custom grok pattern.

(?<day>[0-9]{2})(?<month>[A-Z]{3})(?<year>[0-9]{2})

So, considering the message 0E7 10JUN21 23:37:53.8 Track 1 EZ: 100 EX: 72 Speed: 36 mph and extracting the date and the time from it and making the transformations to parse the data, you will need the following configuration.

filter {
    grok {
        match => {
            "message" => "0E7 (?<day>[0-9]{2})(?<month>[A-Z]{3})(?<year>[0-9]{2}) %{TIME:time} %{GREEDYDATA}"
        }
    }
    mutate {
        capitalize => ["month"]
        add_field => { "datestring" => "%{month} %{day} %{year} %{time}"}
    } 
    date {
        match => ["datestring", "MMM dd yy HH:mm:ss.S"]
        remove_field => ["day","month","year","time"]
    }
}

This will give you the following result:

{
    "@timestamp" => 2021-06-11T02:37:53.800Z,
       "message" => "0E7 10JUN21 23:37:53.8 Track 1 EZ: 100 EX: 72 Speed: 36 mph",
    "datestring" => "Jun 10 21 23:37:53.8"
}

As you can see the @timestamp field has the value of your date string, but relative to UTC as the @timestamp field uses UTC.

Upvotes: 3

Related Questions