CODEWITHSUNDEEP
apache-sparkelasticsearch
Samuel
Samuel

Reputation: 33

ElasticSearch Upsert or Index what is correct my case?

I have two different Spark jobs that feeds the same index on ElasticSearch one fill a certain set of fields and the other auguments the index with the rest of the data.

My question is, should I use index (I think the answer is no, I would loose data from the first job AFAIK) or upsert?

In case it's upsert (which is the one I am playing with right now), it allows FORCE/INTERNAL versioning types, FORCE in theory shouldn't be used, even though it's the one that fits to my needs from a simple testing perspective.

When I try internal I have a version conflict that says that current version is -1 and I am trying to set it to 1. Is it safe to set the version to -1 simply to use INTERNAL instead of FORCE?

Upvotes: 2

Views: 2579

Answers (1)

Viraj Parab
Viraj Parab

Reputation: 91

You can use upsert for your use case read more

For the first case, it will insert the index if it's not their second if the document is there it will just append the values to your document.

Upvotes: 1

Related Questions