CODEWITHSUNDEEP
elasticsearchindexingtypesupdates
raven99
raven99

Reputation: 1421

ElasticSearch: How to update a document type in an existing index

Long story short:

I have created a strict template, created several daily indices and changes one of the document types (%_level type from float to integer) in the template.

After that, I've created the rest of the daily indices.

The problem:

I need to change the document type (%_level type from float to integer) in the old indices to be compatible with the new indices. How can I do this?

Now for the details...

I have the following strict template:

PUT _template/example-template
{
    "order":0,
    "version":200,
    "index_patterns":["example-*"],
    "settings":{
        "index":{
            "number_of_shards":4
         }
    },
    "mappings": {
        "iterations": {
            "dynamic":"strict",
            "properties": {
                "%_average1": {
                    "type":"float"
                },
                "%_average2": {
                    "type":"float"
                },
                "sum": {
                    "type":"integer"
                },
                "%_level": {
                    "type":"float"
                }
            }
        }
    }
}

Several indices were created with this template.

After a while I've realised that we need to change %_level type from float to integer, so I have changed the template into the following:

PUT _template/example-template
{
    "order":0,
    "version":200,
    "index_patterns":["example-*"],
    "settings":{
        "index":{
            "number_of_shards":4
         }
    },
    "mappings": {
        "iterations": {
            "dynamic":"strict",
            "properties": {
                "%_average1": {
                    "type":"float"
                },
                "%_average2": {
                    "type":"float"
                },
                "sum": {
                    "type":"integer"
                },
                "%_level": {
                    "type":"integer"
                }
            }
        }
    }
}

Now the the following indices were created where %_level type is an integer.

But the old indices contains indices with %_level that is float and the new indices are with %_level that is an integer.

I needed to convert the old indices %_level from float to integer so I can build a report with all the indices.

How can I change the %_level in the old indices from float to integer?

Upvotes: 1

Views: 1458

Answers (1)

raven99
raven99

Reputation: 1421

Although you can add and remove document names from an exisitng index by providing the document ID, it is problematic to do so if you want to update the document type, and to apply it on all the index document.

A good solution for this problem will be reindex.

Important:

The reindex process can convert some types implicitly (Numeric Type Casting) and the others explicitly.

In case you need implicit conversion, you can skip section 2

Meaning, apply the following over all the old indices (e.g. example-20191220):

  1. create a new index from example-20191220 that will be called example-20191220-new
  2. add a painless script in the creation process that convert %_level type from float to integer
  3. delete the old index example-20191220
  4. reindex example-20191220-new into example-20191220

Now, the 'updated' index example-20191220 is with the correct %_level type.

The reindex in section 2 should look like the following:

(it is performed and tested on Kibana Dev Tools)

POST _reindex
{
    "source": {
        "index": "example-20191220"
    },
    "dest": {
        "index": "example-20191220-new"
    },
    "script": {
    "lang": "painless", 
    "source": 
        """
        Double num = ctx._source['%_level'];
        if(num != null)
        { ctx._source['%_level'] = num.intValue(); }
        """
    }
}

The reindex in section 4 should look like the following:

(it is performed and tested on Kibana Dev Tools)

POST _reindex
{
    "source": {
        "index": "example-20191220-new"
    },
    "dest": {
        "index": "example-20191220"
    }
}

Upvotes: 2

Related Questions