CODEWITHSUNDEEP
elasticsearch
David Kong
David Kong

Reputation: 638

How to manage "versions" of documents in elasticsearch?

Sometimes, I will have a new set of documents that will "replace" an old set of documents. I would like to still have access to the old set of documents for historical data analysis. However, I would like to primarily be only searching the new set of documents.

One solution to this problem appears to be to create a new index for each grouping. And only search the most recent set of indices. However, then there will be thousands of indices, growing every time an update happens. Is this the optimal strategy or is there another better method?

Since elasticsearch was originally made for search engine which must experience issues like this constaly, I'm surprised there is not a more apparent solution to this problem.

Upvotes: 3

Views: 2714

Answers (1)

jaspreet chahal
jaspreet chahal

Reputation: 9099

Elastic search inbuilt versioning system only keeps track of version numbers on indexed documents , it doesn't maintain historical data.

This you will have to implement yourself. You can do this in multiple ways 1. You can store historical data in document itself

    {
      "id":1,
      "content":"abc",
      "version":[
                    {"timestamp":"2019-10-01","content":""},
                    {"timestamp":"2019-10-02","content":""}
                ]
    }

This will bloat your documents and if content is large will cause performance issue.

2. You can store different version documents in same index with different version numbers.

  {
    "id":1,
    "context":"",
    "version":1
  },
  {
    "id":1,
    "context":"",
    "version":2,
    "Iscurrent":"true"
  }
  1. You can maintain old version in a single index as(2) and latest version in a separate index . So two indexes:- old and new. I think this will be more suitable for your purpse

Upvotes: 3

Related Questions