CODEWITHSUNDEEP
elasticsearchlogstashkibana
Travis
Travis

Reputation: 444

How to bulk create (export/import) indices in elasticsearch?

I'm trying to upgrade our ELK stack from 1.x > 5.x following the re-index from remote instructions. I'm not sure of how to export a list of the indices that I need to create and then import that list into the new instance. I've created a list of indices using this command, both with "pretty," and without, but I'm not sure which file format to use as well as what to next do with that file.

The create index instructions don't go into how to create more than one at a time, and the bulk instructions only refer to creating/indexing documents, not creating the indices themselves. Any assistance on how to best follow the upgrade instructions would be appreciated.

I apparently don't have enough reputation to link the "create index" and "bulk" instructions, so apologies for that.

Upvotes: 2

Views: 2242

Answers (2)

Val
Val

Reputation: 217544

With a single curl command you could create an index template that will trigger the index creation at the time the documents hit your ES 5.x cluster.

Basically, this single curl command will create an index template that will kick in for each new index created on-the-fly. You can then use the "reindex from remote" technique in order to move your documents from ES 1.x to ES 5.x and don't worry about index creation since the index template will take care of it.

curl -XPUT 'localhost:9200/_template/my_template' -H 'Content-Type: application/json' -d'
{
  "template": "*",
  "settings": {
     "index.refresh_interval" : -1,
     "index.number_of_replicas" : 0
  }
}
'

Upvotes: 2

Travis
Travis

Reputation: 444

Was able to accomplish this with a formatted list of indices created via an index list fed through sed, then feeding that file through the following script:

#! /bin/bash

while read some_index; do
curl -XPUT "localhost:9200/$some_index?pretty" -d'
{
    "settings" : {
        "index" : {
            "refresh_interval" : -1,
            "number_of_replicas" : 0
        }
    }
}'
sleep 1
done <$1

If anyone can point me in the direction of any pre-existing mechanisms in Elasticsearch, though, please do.

Upvotes: 0

Related Questions