CODEWITHSUNDEEP
jsonelasticsearch
Sarah Mandana
Sarah Mandana

Reputation: 1513

Synchronizing JSON data in Elastic Search

I have a JSON file data which is updated continuously (after every hour). What would be the best approach to synchronize this data with elastic search's index?

What I am doing at the moment? I just delete and add index again and it is not a good approach for live version of my application.

What best approach would you people suggest?

Upvotes: 1

Views: 236

Answers (1)

Tomáš Linhart
Tomáš Linhart

Reputation: 10210

I don't think that your approach is completely wrong. However, if you need to retain availability, it's best to use index aliases. Think of alias as a symlink to the index. From the documentation:

The index aliases API allows aliasing an index with a name, with all APIs automatically converting the alias name to the actual index name.

Initially, you index your documents (from the JSON file) to some index, let's say data-2019-04-29-08-00 (named based on the date and time), and create an alias data for this index. You target all your queries against the data alias. When your data (JSON file) change the next hour, you index the data into a new index, data-2019-04-29-09-00 and change the data alias to point to this new index. Now all the queries will work with the new data. This way, no matter how long indexing the documents take, you won't suffer any outage as changing the alias is a really fast operation.

Upvotes: 1

Related Questions