CODEWITHSUNDEEP
elasticsearch
pmckeown
pmckeown

Reputation: 4219

ElasticSearch - Configuration to Analyse a document on Indexing

In a single request, I want to retrieve documents from a SOR, store them in ElasticSearch, and then search those documents using the ES search API.

There seems to be some lag from the time the document is indexed and the time it is analyzed and ready to be searched.

Is there any way to configure ES to not return from the request to index a document until the analyzer has analyzed it and so that it can immediately be searched?

Upvotes: 0

Views: 74

Answers (2)

Avinash Kumar Pandey
Avinash Kumar Pandey

Reputation: 762

The refresh API, as suggested in the accepted answer, is heavy in nature and you may not want to call this API after every index operation, if you are going to do a significant number of indexing operations.

What happens under the hood is that the translog maintained by elasticsearch is written to the in memory segment which elasticsearch maintains. This operations is best left to the discretion of elasticsearch, however, there are some configuration parameters you can play around with.

There is an alternative approach you can take, it may or may not suit your specific use case, but here it goes.

Query the index/_stats/refresh api and retrieve the status of refresh from there, index your document and then keep performing the same stats query again. If the version has increased since your indexing time, it means you are good for searching your document.

https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html

Upvotes: 1

Val
Val

Reputation: 217554

Elasticsearch is "near real-time" by nature, i.e. all indices are refreshed every second (by default). While it may seem enough in a majority of cases, it might not, such as in your case.

If you need your documents to be available immediately, you need to refresh your indices explicitly by calling

POST /_refresh

or if you only want to refresh one index

POST /my_index/_refresh

The refresh needs to happen after the indexing call returned and before the search call is sent off.

Note that doing this on every document indexing will hurt the performance of your system. It might be better to make your application aware of the near real-time nature of ES and handle this on the client-side.

Upvotes: 1

Related Questions