CODEWITHSUNDEEP
elasticsearch
Espresso
Espresso

Reputation: 5739

How to move elasticsearch index using file system?

Usecase:

I have created es-indexes: mywebsiteindex-yyyymmdd , mysharepointindex-yyyymmdd in my laptop/dev machine. I want to export/zip that index as a file. The file may be migrated by someone who has credentials to target machine. And the zip/file may be imported to target-elastic folder.

You can abstract the words 'machine' 'folder' 'zip' in the above. Focus is 'transfer index as a file and reimport at target which I may not have access through http/tcp/ftp/ssh'.

Is there any python/other script out there that can export-from-source and import-to-target? A script that hides internal complexities of node/cluster count differences between dev/prod etc, and just move index.

Note: I already referred to the below page, so no need to reiterate the same

https://www.elastic.co/guide/en/cloud/current/ec-migrate-data.html

Upvotes: 2

Views: 1282

Answers (2)

Joel Aelwyn
Joel Aelwyn

Reputation: 357

While I don't have a script for it, I have migrated a variety of indices by using the built-in snapshot APIs. The trick is that you want to look at the REST API documentation rather than at the cloud-focused documentation.

More specifically: you can create a snapshot repository with a type of "fs" to use local filesystem storage, then export a snapshot of the index you care about to that repository. Once you do so, it is safe to "bundle up" the whole thing and send it to another machine for restoration (the target Elastic instance will also need a repository defined, so that it has a path where it can look for the snapshot information). Don't forget to set the configuration option that controls where repositories are located, of course.

Upvotes: 0

leandrojmp
leandrojmp

Reputation: 7473

There are some options:

  • You can use the snapshot and restore api to create a snapshot of your index and restore it in your new instance. (recommended way)
  • You can use the reindex api in your new instance to reindex your index from remote.
  • You can use Logstash with your old instance as an input and your new instance as the output.
  • And you can write a script/application using one of the supported clients to query your index, export to a file, read that file and import in your new instance. (logstash can also do that).

But you can't move your data files, this is not supported nor recommended by Elastic.

Upvotes: 1

Related Questions