CODEWITHSUNDEEP
elasticsearch
Andrey Popov
Andrey Popov

Reputation: 163

Elasticsearch compare two indexes

I have two different indexes - index1 and index2 In index1 I have documents which IDs match index2 (don't ask the reason it is done so, but the point is it is like this). So IDs amount and values must match in index1 and index2. Sometimes i have missing ID in one of the tables... so the question is: Is there any way to compare IDs of index1 and index2? For example i can count and amount of records in both indexes and if they mismatch can compare ID of each document in Index1 with ID in Index2 (check if it exists), but it will be really slow so I'm looking for the easier solution:)

Any idea will be appreciated!

Upvotes: 3

Views: 7835

Answers (4)

Adam Clark
Adam Clark

Reputation: 61

I accomplished this by using the * syntax

  • index 1: 'foo-bar'
  • index 2: 'foo-baz'
  • etc. and finally
  • comparison index: 'foo-*'

Upvotes: 0

Gazouille
Gazouille

Reputation: 11

One idea to find documents that are present in only one of the indexes could be to reindex both indices into a new third one and then search in that index for documents that have version > 1.

Upvotes: 1

notzippy
notzippy

Reputation: 508

I am struggling with this as well - you would think this would be easy. My thoughts are

  1. Add in a property to track when last found the item - say FoundState
  2. Sweep update index1 type x, set FoundState to NotFound
  3. As you populate the new index2 update the matching id in index1 showing it was found in the new index (FoundState Found). You could even update the new item showing it was found or not.

Then you can filter on index2 for new items, or items found, and index1 can be filtered based on the FoundState for missing items

The extra work updating the items in the first index is a pain but the only way I can see in doing it

Upvotes: 0

Sebastian
Sebastian

Reputation: 2550

You could add a new bool field "in_index2" to index1 and count that bool field.

I don't like that solution, but it should be working.

Upvotes: 0

Related Questions