CODEWITHSUNDEEP
elasticsearchlogstashelastic-stack
KidWithAComputer
KidWithAComputer

Reputation: 331

How to get ElasticSearch output?

I want to add my log document to ElasticSearch and, then I want to check the document in the ElasticSearch. Following is the conntent of the log file :

Jan  1 06:25:43 mailserver14 postfix/cleanup[21403]: BEF25A72965: message-id=<[email protected]>
Feb  2 06:25:43 mailserver15 postfix/cleanup[21403]: BEF25A72999: message-id=<[email protected]>
Mar  3 06:25:43 mailserver16 postfix/cleanup[21403]: BEF25A72998: message-id=<[email protected]>

I am able to run my logstash instance with following logstast configuration file :

input {
  file {
    path => "/Myserver/mnt/appln/somefolder/somefolder2/testData/fileValidator-access.LOG"
    start_position => "beginning"
    sincedb_path => "/dev/null"
  }
}

filter {
  grok {
    patterns_dir => ["/Myserver/mnt/appln/somefolder/somefolder2/logstash/pattern"]
    match => { "message" => "%{SYSLOGBASE} %{POSTFIX_QUEUEID:queue_id}: %{GREEDYDATA:syslog_message}" }
  }
}

output{
    elasticsearch{
       hosts => "localhost:9200"
       document_id => "test"
       index => "testindex"
       action => "update"
    }
stdout { codec => rubydebug }
}

I have define my own grok pattern as :

POSTFIX_QUEUEID [0-9A-F]{10,11}

When I am running the logstash instance, I am successfully sending the data to elasticsearch, which gives following output :enter image description here

Now, I have got the index stored in elastic search under testindex, but when I am using the curl -X GET "localhost:9200/testindex" I am getting following output :

{
  "depositorypayin" : {
    "aliases" : { },
    "mappings" : { },
    "settings" : {
      "index" : {
        "creation_date" : "1547795277865",
        "number_of_shards" : "5",
        "number_of_replicas" : "1",
        "uuid" : "5TKW2BfDS66cuoHPe8k5lg",
        "version" : {
          "created" : "6050499"
        },
        "provided_name" : "depositorypayin"
      }
    }
  }
}

This is not what is stored inside the index.I want to query the document inside the index.Please help. (PS: please forgive me for the typos)

Upvotes: 1

Views: 930

Answers (2)

KidWithAComputer
KidWithAComputer

Reputation: 331

Actually I have edited my config file whic look like this now :

input {
. . .
}

filter {
 . . .
}

output{
    elasticsearch{
       hosts => "localhost:9200"
       index => "testindex"
    }

}

And now I am able to get fetch the data from elasticSearch using

curl 'localhost:9200/testindex/_search'

I don't know how it works, but it is now. can anyone explain why ?

Upvotes: 0

Joe Waller
Joe Waller

Reputation: 1989

The API you used above only returns information about the index itself (docs here). You need to use the Query DSL to search the documents. The following Match All Query will return all the documents in the index testindex:

curl -X GET "localhost:9200/testindex/_search" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}
'

Upvotes: 1

Related Questions