CODEWITHSUNDEEP
elasticsearchspring-data-elasticsearch
Amit Patel
Amit Patel

Reputation: 134

Best way to synchronize Elasticsearch with Mysql

I am using elasticsearch in my spring web mvc project (spring-data-elasticsearch) and to synchronize with database (MySQL).

I am indexing the document from my app, if any new entity going to add in db tables at the same time, from service layer, I request to index this document to elasticsearch also.

Both db tables and elasticsearch index have same data and to delete and update operation on I am using same concept, performing the change operation on elasticsearch and db table, it is working fine.

Now I want to know what will be the disadvantages of this approach.

Or is there any best way to make our elasticsearch index up to date from db. I used logstash but what about the deleted entities

Upvotes: 9

Views: 6177

Answers (1)

ka3boss
ka3boss

Reputation: 26

The disadvantage of Synchronous indexation is there is no retry if there is an error while creating index data.

At your place i will create a cronjob/batch ( for trigger it depends how much data are updated and how important is the update of index ) and this job will have execution status with logs

you will have the clear idea about your index and no missing data

And for indexes you can a FULL index mode & an UPDATE indexes mode ( you should add an update date on your tables )

Indexing strategy you have two phases and you can choose TWO_PHASES : you need a master & slave ==> while executing indexing on master the slave will respond to requests and when the indexing is over you synchronize DIRECT_MODE : drop index & create new one

Upvotes: 1

Related Questions