CODEWITHSUNDEEP
elasticsearchopensearch
Rajesh Kumar
Rajesh Kumar

Reputation: 21

Index a new field existing in _source

I have JSON stored in _source with n number of fields but not all of then are indexed or present in index mapping. If i need to index a new field or add a new field to the index/index mapping already present in _source, what's the best way to do this ?

The only approach i get to know is re-index to a new index with updated index mapping.

This also bring to next curiosity is there a way re-index data in-place or on the same index.

We keep dynamic mapping false so we index only selected fields.

Upvotes: 1

Views: 1279

Answers (1)

Val
Val

Reputation: 217514

If you have dynamic mapping set to false, which is usually a good idea to prevent mapping explosion and stay in control of your mapping, and you want to add a new field to your index, you can simply update your index mapping using the update mapping API and then update your index in place using the update by query API.

For instance, let's say you have the following index with a single field and dynamic false:

PUT test
{
  "mappings": {
    "dynamic": false,
    "properties": {
      "field1": {
        "type": "keyword"
      }
    }
  }
}

Then you index the following document:

PUT test/_doc/1
{
  "field1": "Lorem"
}

And a while later your requirements change and you need to add a second field to your documents, such as the one below:

PUT test/_doc/2
{
  "field1": "Lorem",
  "field2": "Ipsum"
}

You can search by field1 and find both documents when searching for Lorem. However, searching by field2 will not yield anything because the field has been ignored. Yet, it is available in your source.

What you need to do is to update your mapping in place to add your new field, like this:

PUT test/_mapping
{
  "properties": {
    "field1": {
      "type": "keyword"
    },
    "field2": {
      "type": "keyword"
    }
  }
}

At this point, field2 in all newly indexed documents will be properly indexed. But field2 in all previously indexed documents are still not indexed. For that, you need to update your data in place using the following update by query call:

POST test/_update_by_query

When the call returns, you'll be able to search by field2 in all documents, whether old and new ones. Searching field2 for Ipsum will return the document with ID above.

Upvotes: 2

Related Questions