CODEWITHSUNDEEP
elasticsearchlogstash
Al-Punk
Al-Punk

Reputation: 3660

ElasticSearch assign own IDs while indexing with LogStash

I am indexing a large corpora of information and I have a string-key that I know is unique. I would like to avoid using the search and rather access documents by this artificial identifier.

Since the Path directive is discontinued in ES 1.5, anyone know a workaround to this problem!?

My data look like:

{unique-string},val1, val2, val3...
{unique-string2},val4, val5, val6...

I am using logstash to index the files and would prefer to fetch the documents through a direct get, rather than through an exact-match.

Upvotes: 1

Views: 86

Answers (1)

Val
Val

Reputation: 217274

In your elasticsearch output plugin, just specify the document_id setting with a reference to the field you want to use as id, i.e. the one named 1 in your csv filter.

input {
    file {...}
}
filter {
    csv{
        columns=>["1","2","3"]
        separator => ","
    }
}
output {
 elasticsearch {
     action => "index"
     host => "localhost"
     port => "9200"
     index => "index-name" 
     document_id => "%{1}"              <--- add this line
     workers => 2
     cluster => "elasticsearch-cluster"
     protocol => "http"
 }
}

Upvotes: 2

Related Questions