Reputation: 11
if [CREATION_DATE] == ""
{
mutate {
convert => [ "CREATION_DATE", "string" ]
}
}
else
{
date {
locale => "en"
match => [ "CREATION_DATE", "dd-MMM-yy hh.mm.ss.SSS a"]
target => "CREATION_DATE"
}
}
if [SUBMITTED_DATE] == ""
{
mutate {
convert => [ "SUBMITTED_DATE", "string" ]
}
}
else
{
date {
locale => "en"
match => [ "SUBMITTED_DATE", "dd-MMM-yy hh.mm.ss.SSS a"]
target => "SUBMITTED_DATE"
}
}
if [LAST_MODIFIED_DATE] == ""
{
mutate {
convert => [ "LAST_MODIFIED_DATE", "string" ]
}
}
else
{
date {
locale => "en"
match => [ "LAST_MODIFIED_DATE", "dd-MMM-yy hh.mm.ss.SSS a"]
target => "LAST_MODIFIED_DATE"
}
}'
am getting output if i have all three (CREATION_DATE,SUBMITTED_DATE,LAST_MODIFIED_DATE) in date format.If any is STRING am not getting that log file in my input. for ex: my input is
12-JUL-13 11.33.56.259 AM,12-JUL-13 03.59.36.136 PM,12-JUL-13 04.00.05.584 PM
14-JUL-13 11.33.56.259 AM,11-JUL-13 04.00.05.584 PM
my output will come successfully for
12-JUL-13 11.33.56.259 AM,12-JUL-13 03.59.36.136 PM,12-JUL-13 04.00.05.584 PM
but NOT FOR 2nd line
In Simple,Logstash is indexing only when three if clauses have dates. Help me out.THanks in advance!!
Upvotes: 0
Views: 785
Reputation: 11
Issue is solved when i change input format. (New Format) 11-JUL-13 06.36.33.425000000 PM,13-JUL-13 06.36.33.425000000 PM,, instead of (Old format)11-JUL-13 06.36.33.425000000 PM,13-JUL-13 06.36.33.425000000 PM,"", But My ques is still open. I posted this because this solution might be useful to some. Thanks!!!
Upvotes: 0
Reputation: 4089
The issue with your if statements is pointed out by the comments by @Fairy and @alain-collins.
if [CREATION_DATE] == ""
Does not check if that field exists, it checks if it is an empty string.
Instead you could use a regex check to see if there is any content in the field using:
if [CREATION_DATE] =~ /.*/
and perform your date filter when this returns true.
Upvotes: 1