CODEWITHSUNDEEP
elasticsearchlogstashlogstash-groklogstash-configuration
AleGallagher
AleGallagher

Reputation: 1893

Logstash - Add fields from the log - Grok

I'm learning logstash and I'm using Kibana to see the logs. I would like to know if is there anyway to add fields using data from message property.

For example, the log is like this:

@timestamp:December 21st 2016, 21:39:12.444 port:47,144  
appid:%{[path]} host:172.18.0.5 levell:level message:
{"@timestamp":"2016-12-22T00:39:12.438+00:00","@version":1,"message":"Hello","logger_name":"com.empresa.miAlquiler.controllers.UserController","thread_name":"http-nio-7777-exec-1","level":"INFO","level_value":20000,
"HOSTNAME":"6f92ae402cb4","X-Span-Export":"false","X-B3-SpanId":"8f548829e9d18a8a","X-B3-TraceId":"8f548829e9d18a8a"}

My logstash conf is like:

filter {
grok {
match => {
  "message" =>
  "^%{TIMESTAMP_ISO8601:timestamp}\s+%{LOGLEVEL:level}\s+%{NUMBER:pid}\s+---\s+\[\s*%{USERNAME:thread}\s*\]\s+%{JAVAFILE:class}\s*:\s*%{DATA:themessage}(?:\n+(?<stacktrace>(?:.|\r|\n)+))?$"
}
}
 date {
match => [ "timestamp" , "yyyy-MM-dd HH:mm:ss.SSS" ]
}
mutate {
remove_field => ["@version"]
add_field => {
  "appid" => "%{[path]}"
}
add_field => {
  "levell" => "level"
}

} }

I would like to take level (in the log is INFO), and message (in the log is Hello) and add them as fields.

Is there anyway to do that?

Upvotes: 1

Views: 8671

Answers (1)

Kulasangar
Kulasangar

Reputation: 9454

What if you do something like this using mutate:

filter { 
  mutate { 
    add_field => ["newfield", "%{appid} %{levell}"] <-- this should concat both your appid and level to a new field
  } 
}

You might have a look at this thread.

Upvotes: 3

Related Questions