CODEWITHSUNDEEP
searchluceneelasticsearch
Carson
Carson

Reputation: 17751

Can ElasticSearch delete all and insert new documents in a single query?

I'd like to swap out all documents for a specific index's type. I'm thinking about this like a database transaction, where I'd:

  1. Delete all documents inside of the type
  2. Create new documents
  3. Commit

It appears that this is possible with ElasticSearch's bulk API, but is there a more direct way?

Upvotes: 0

Views: 1532

Answers (1)

Paige Cook
Paige Cook

Reputation: 22555

Based on the following statement, from the elasticsearch Delete by Query API Documentation:

Note, delete by query bypasses versioning support. Also, it is not recommended to delete "large chunks of the data in an index", many times, it’s better to simply reindex into a new index.

You might want to reconsider removing entire types and recreating them from the same index. As this statement suggests, it is better to simply reindex. In fact I have a scenario where we have an index of manufacturer products and when a manufacturer sends an updated list of products, we load the new data into our persistent store and then completely rebuild the entire index. I have implemented the use of Index Aliases to allow for masking the actual index being used. When products changes occur a process is started to rebuild the new index in the background (a process that currently takes about 15 minutes) and then switch the alias to the new index once the data load is complete and delete the old index. So this is completely seamless and does not cause any downtime for our users.

Upvotes: 2

Related Questions