CODEWITHSUNDEEP
pythonelasticsearchpyes
JChao
JChao

Reputation: 2319

Elasticsearch is missing data when querying after insert

I have a test that does the following:

  1. create an index on ES using pyes
  2. insert mapping
  3. insert data
  4. query the data to check if it fits the expected result

However when I run this test, sometimes it would find no result, and sometimes it would find something but some data is missing. The interesting thing is this only happens when I run the test automatically. If I type in the code ling-by-line, everything works as expected. I've tested it 3 times manually and it works without failure.

Sometimes I've even received this message:

NoServerAvailable: list index out of range

It seems like the index was not created at all

I've pinged my ES address and everything looks correct. There was no other error msg found anywhere.

I thought it would be because I was trying to get the data too fast after the insert, as documented here: Elastic Search No server available, list index out of range

(check first comment in the accepted answer)

However, even if I make it delay for 4 seconds or so, this problem still happens. What concerns me is the fact that sometimes only SOME data is missing, which is really strange. I thought it would be either finding it all or missing it all.

Anyone got similar experience that can shine some light on this issue? Thanks.

Upvotes: 1

Views: 1945

Answers (1)

Hatim Stovewala
Hatim Stovewala

Reputation: 1251

Elasticsearch is Near Realtime(NRT). It might take upto 1 second to make your recently indexed document be visible/available for search. To make your recently indexed document available instantly for searching, you can append ?refresh=wait_for at the end of the index document command. For e.g.,

POST index/type?refresh=wait_for
{
  "field1":"test",
  "field2":2,
  "field3":"testing"
}

?refresh=wait_for will forcibly refresh your index to make the recently indexed document available for search. Refer ?refresh.

Upvotes: 0

Related Questions