CODEWITHSUNDEEP
elasticsearch
Bilberryfm
Bilberryfm

Reputation: 537

Add new fields in 10 million documents in ElasticSearch

I have ElasticSearch index that stores more than 10 mln documents. I need to iterate over all index and add new field and value for each document. I know about two approaches: 1. Use scroll search to get all results and use BULK api to copy all documents into new index [adding new fields while copying]. After all documents are copied to new index - switch aliases. 2. Use scroll search and bulk API to update all documents in existed index (do not need to copy to new index and switching aliases).

I do not understand, why first approach is recommended? Can I just use 2 approach? Is there any possibility that while running program with 2 approach, something bad can happen to the index? Index that I need to update has live prod data and I worried that customers may be affected in some way

Upvotes: 3

Views: 4071

Answers (1)

prasad kp
prasad kp

Reputation: 861

Use the script painless 

POST /mysqltest/_update_by_query
{
  "query": {
    "match_all": {}
  },
 "script": { "inline": "ctx._source.category = \"10\";" } --> category is new key that to be added in all documents
}

Upvotes: 8

Related Questions