CODEWITHSUNDEEP
logstashlogstash-grok
Sarde
Sarde

Reputation: 688

Logstash Grok pattern for session id and null values

Question 1 -

56dd573d.5edd this is my session id, i have grok filter like

%{WORD:session_id}.%{WORD:session_id} - this will read the session id and output will look like this

 "session_id": [
    [
      "56dd573d",
      "5edd"
    ]
  ]

Is there any way where i can get output something like 

  "session_id": [
    [
      "56dd573d.5edd"
    ]
  ]

I just need it in single field

Question 2 -

2016-03-08 06:48:15.477 GMT

this is a line from my log entry, i have used 

%{DATESTAMP:log_time} %{WORD}

grok filter to read this date, here i simply want to drop or ignore the GMT

Is there any special pattern to ignore the next word from the log line which is not useful ?

Updated

Question 3 - How do i handle null value, its after GMT 

2016-03-07 10:26:05 GMT,,

This is my postgresql log entry

2016-03-08 06:48:15.477 GMT,"postgres","sugarcrm",24285,"[local]",56dd573d.5edd,4,"idle",2016-03-07 10:26:05 GMT,,0,LOG,00000,"disconnection: session time: 20:22:09.928 user=postgres database=sugarcrm host=[local]",,,,,,,,,""

Note - null value may be in "" or ,,

Answer for question 3

I found the solution for handling ,,

Below is configuration for handling ,, value by replacing 0 with it

input {
  file {
    path => "/var/log/logstash/postgres.log"
    start_position => "beginning"
    type => "postgres"
  }
}

filter {
  mutate {
    gsub => [
      "message", "^,", "0,",
      "message", ",,", ",0,",
      "message", ",,", ",0,",
      "message", ",,", ",0,",
      "message", ",$", ",0"
    ]
  }
grok {
        match => ["message","%{GREEDYDATA:msg1}"]
}
}

output {
  stdout { codec => rubydebug }
}

Reference - http://comments.gmane.org/gmane.comp.sysutils.logstash.user/13842

But i am trying for "" null value i tried below configuration but i am getting configuration error 

filter {   mutate {
    gsub => [
      "message", "^,", "0,",
      "message", ",,", ",0,",
      "message", ",,", ",0,",
      "message", ",,", ",0,",
      "message", ",$", ",0",
      "message", "^\""  "null\""
      "message", """" ""null""
      "message", """" ""null""
      "message", ""$", ""null"
    ]   }

I need to replace "" with null

Upvotes: 1

Views: 4490

Answers (1)

W Khan
W Khan

Reputation: 120

Regarding question 1. It separates the two because essentially what youre asking it to do it add another value to session_id. You want something like:

(?<session_ID>(%{WORD}.%{WORD}))

Try it out on https://grokdebug.herokuapp.com/ . Where you can test your patterns. I The above isnt the greatest of solutions, but I dont have enough information about the rest of the message. Because if you know more, you can throw away the WORD match. If it is a structured session_ID with fixed length, for example, you can do:

(?<session_ID>([a-zA-Z0-9]{1,8}\.)[a-zA-Z0-9]{1,4})

Regarding the second question. I would hard code it for a quick hack:

%{DATESTAMP:log_time} GMT

give some more information and we can give a better more specific answer. The above should work, but there are several ways to skin a cat!

Upvotes: 2

Related Questions