CODEWITHSUNDEEP
elasticsearchlogstashlogstash-grok
tricky
tricky

Reputation: 1553

Automatically parse logs fields with Logstash

Let's say I have this kind of log :

Jun 2 00:00:00 192.168.14.4 date=2016-06-01 time=23:56:05 devname=POPB-FW-01 devid=FG1K2D3I14800220 logid=1059028704 type=utm subtype=app-ctrl eventtype=app-ctrl-all level=information vd="root" appid=40568 user="" srcip=10.20.4.35 srcport=52438 srcintf="VRF-PUBLIC" dstip=125.209.230.238 dstport=443 dstintf="OUT" proto=6 service="HTTPS" sessionid=424666004 applist="Monitor-all" appcat="Web.Others" app="HTTPS.BROWSER" action=pass hostname="lcs.naver.com" url="/" msg="Web.Others: HTTPS.BROWSER," apprisk=medium

So with this code below, I can regex the timestamp and the ip in future elastic fields :

filter {
    grok {
    match => {"message" => "%{SYSLOGTIMESTAMP:timestamp} %{client}" }
    }
}

Now, how do I automatically get fields for the rest of the log ? Is there a simple way to say :

The thing before the "=" is the field name and the thing after is the value.

So I can obtain a JSON for elastic index with many fields for each log line :

{

    "path" => "C:/Users/yoyo/Documents/yuyu/temp.txt",
    "@timestamp" => 2017-11-29T10:50:18.947Z,
    "@version" => "1",
    "client" => "192.168.14.4",
    "timestamp" => "Jun  2 00:00:00",
    "date" => "2016-06-01",
    "time" => "23:56:05",
    "devname" => "POPB-FW-01 ",
    "devid" => "FG1K2D3I14800220",
    etc,...

}

Thanks in advance

Upvotes: 0

Views: 98

Answers (1)

tricky
tricky

Reputation: 1553

Okay, I am really dumb

It was easy, rather than search on google, how to match equals, I just had to search key value matching with logstash.

So I just have to write :

filter {
   kv {
   }
}

And it's done !

Sorry

Upvotes: 1

Related Questions