CODEWITHSUNDEEP
elasticsearchindexing
Vishnu Ranganathan
Vishnu Ranganathan

Reputation: 1355

How to stop auto reindexing in elastic search if any update happens?

I am having a big use case with elasticsearch which has millions of records in it.

I will be updating the records frequently, say 1000 records per hour.

I don't want elastic search to reindex for my every update.

I am planning to reindex it on weekly basis.

Any Idea how to stop auto-reindex while update ?

Or any other better suggestion is welcome . Thanks in advance :)

Upvotes: 2

Views: 1505

Answers (1)

Amit
Amit

Reputation: 32376

Elasticsearch(ES) update an existing doc in below manner. 1. Deletes the old doc. 2. Index a new doc with the changes applied to it.

According to ES docs :-

In Elasticsearch, this lightweight process of writing and opening a new segment is called a refresh. By default, every shard is refreshed automatically once every second. This is why we say that Elasticsearch has near real-time search: document changes are not visible to search immediately, but will become visible within 1 second.

Note that these changes will not be visible/searchable until ES commits/flush these changes to disk cache and disk,which is control by soft-commit(es refresh interval, which is by default 1 second) and hard-commit(which actually write the document to disk, which prevent it being lost permanently and costly affair than a soft-commit).

You need to make sure, you tune your ES refresh interval, and do proper load testing, as setting it very low and very high has its own pros and cons.

for example setting it very less for example 1 second and if you have too many updates happening than it has a performance hit and it might crash your system. Also setting it very high for example 1 hour means you now don't have a NRT(near real time search) and during that time if your memory could contain again millions of doc(depending on your app) and can cause out of memory error, also committing on such a large memory is a very costly affair.

Upvotes: 3

Related Questions