CODEWITHSUNDEEP
elasticsearch
donkeyx
donkeyx

Reputation: 616

Should I be using database ID's as Elastic ID's

I am new to elastic and starting to sync my database tables into elastic indexes. I have started by using the table ID(UUID) as the elastic id, but I am starting to wonder if this is a mistake in terms of performance or flexibility in the long term? Any advice would be appreciated.

Upvotes: 2

Views: 1119

Answers (1)

mgaert
mgaert

Reputation: 2388

I think this approach should actually be a best practice. When you update data in your ES index from the (changed) DB, you can address the document directly.

It has worked great for us to use the _bulk update API, which requires an explicit id per item.

On every change on the DB side, we enqueue change notifications, the changed object gets JSON-serialized and sent to ES, asynchronously, and in larger batches. That is making a huge performance difference. Search performance, on the other side, does not depend on the length of the _id AFAIK, not even when you look up by _id. So your DB UUID should be just fine. Especially since _ids can be alphanumeric, they are not limited to just numbers.

Having a 1:1 relationship via _id between the ES result and your system of record (I assume that's what your DB is for) is advantageous also for transparency purposes. In any case, you want to store the database ID as some field, ideally indexed, at least, to help you understand where that document came from.

So, rather than creating your own ID field, you may as well use the built-in _id field right away, with your DB-supplied data.

Upvotes: 2

Related Questions