CODEWITHSUNDEEP
regexlogstashlogstash-groklogstash-configuration
omer khalid
omer khalid

Reputation: 895

How to format date in Logstash Configuration

I am using logstash to parse log entries from an input log file.

LogLine:

TID: [0] [] [2016-05-30 23:02:02,602]  INFO {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService} -  Configured Registry in 572ms {org.wso2.carbon.registry.core.jdbc.EmbeddedRegistryService}

Grok Pattern:

TID:%{SPACE}\[%{INT:SourceSystemId}\]%{SPACE}\[%{DATA:ProcessName}\]%{SPACE}\[%{TIMESTAMP_ISO8601:TimeStamp}\]%{SPACE}%{LOGLEVEL:MessageType}%{SPACE}{%{JAVACLASS:MessageTitle}}%{SPACE}-%{SPACE}%{GREEDYDATA:Message}

My grok pattern is working fine. I am sending these parse entries to an rest base api made by myself.

Configurations:

output {
    stdout { }
     http {
        url => "http://localhost:8086/messages"
        http_method => "post"
        format => "json"
        mapping => ["TimeStamp","%{TimeStamp}","CorrelationId","986565","Severity","NORMAL","MessageType","%{MessageType}","MessageTitle","%{MessageTitle}","Message","%{Message}"]
    }
}

In the current output, I am getting the date as it is parsed from the logs:

Current Output:

{ 
"TimeStamp": "2016-05-30 23:02:02,602"
}

Problem Statement:

But the problem is that my API is not expecting the date in such format, it is expecting the date in generic xsd type i.e datetime format. Also, as mentioned below:

Expected Output:

{ 
"TimeStamp": "2016-05-30T23:02:02:602"
}

Can somebody please guide me, what changes I have to add in my filter or output mapping to achieve this goal.

Upvotes: 0

Views: 968

Answers (1)

Val
Val

Reputation: 217554

In order to transform

2016-05-30 23:02:02,602

to the XSD datetime format

2016-05-30T23:02:02.602

you can simply add a mutate/gsub filter in order to replace the space character with a T and the , with a .

filter {
  mutate {
    gsub => [
      "TimeStamp", "\s", "T",
      "TimeStamp", ",", "."
    ]
  }
}

Upvotes: 1

Related Questions