CODEWITHSUNDEEP
elasticsearchlogstashkibana
vbNewbie
vbNewbie

Reputation: 3345

elasticsearch message field value

I am sending json messages to logstash getting indexed by elasticsearch and managed to setup the UI dashboard in Kibana. I would like to filter the data by the message fields and cannot figure out how or where to do this. An example of my message:

{"message":"{"pubDate":"2014-02-25T13:09:14",
 "scrapeDate":"2014-02-5T13:09:26",
 "Id":"78967",
 "query":"samsung S5",
 "lang":"en"}

Right now it counts all these messages coming in but I need to get each message filtered by the fields itself for example like Id or lang or query. Does this have to be done in the config file or can it be created in Kibana interface.

Upvotes: 2

Views: 4416

Answers (1)

Ban-Chuan Lim
Ban-Chuan Lim

Reputation: 7890

First, I assume your json messages is 

{
   "pubDate":"2014-02-25T13:09:14",
   "scrapeDate":"2014-02-5T13:09:26",
   "Id":"78967",
   "query":"samsung S5",
   "lang":"en"
}

When you send your message to logstash, you need to specify the codec to json. As show in the configuration below: 

input {
    stdin {
            codec => json
    }
}

output {
    elasticsearch {
            cluster => "abc"
    }
}

Logstash will parsing your message to different field, like the output: 

{
   "pubDate" => "2014-02-25T13:09:14",
"scrapeDate" => "2014-02-5T13:09:26",
        "Id" => "78967",
     "query" => "samsung S5",
      "lang" => "en",
  "@version" => "1",
"@timestamp" => "2014-02-26T01:36:15.336Z",
      "host" => "AAAAAAAAAA"
 }

When you show this data in Kibana, You can use fieldname:value to query and filter what you need. For example, you can query all message with lang:en.

Upvotes: 4

Related Questions