CODEWITHSUNDEEP
restelasticsearchlogstashkibana
James Forbes
James Forbes

Reputation: 309

How to use expressions in Elasticsearch REST request body?

Is it possible to use expressions within Elasticsearch REST API body? I am aware that you can use date-math expressions in the REST query string but I can't seem to find an example of it working in the REST request body.

My goal is to create an ILM policy and index template that will rollover the indices and account for the daily change of the date on the index name. An example of an index name would be i.e. index-2020.12.17

PUT /_index_template/index_template
{
  "index_patterns": ["%3Cindex-%7Bnow%2Fd%7D%3E"],                 
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1,
      "index.lifecycle.name": "index_policy",      
      "index.lifecycle.rollover_alias": "%3Cindex-%7Bnow%2Fd%7D%3E"    
    },
    "aliases": {
    "%3Cindex-%7Bnow%2Fd%7D%3E": {
        "is_write_index": true
    }
  }
}

Upvotes: 1

Views: 314

Answers (2)

Val
Val

Reputation: 217254

This is not the way ILM works.

  • index_patterns should simply by a wildcard that matches your index names
  • rollover_alias should be a constant string that applications write to
  • you don't need to define any alias for ILM to work, it's what rollover_alias is for.

So your index template definition should simply be like this:

PUT /_index_template/index_template
{
  "index_patterns": ["index-*"],                 
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1,
      "index.lifecycle.name": "index_policy",      
      "index.lifecycle.rollover_alias": "index-write"    
    },
    "aliases": {
    }
  }
}

Then you can bootstrap the initial index with:

PUT index-2020.12.17-000001
{
  "aliases": {
    "index-write": {
      "is_write_index": true
    }
  }
}

Then your client application (Filebeat, Logstash, etc) can simply send their data to index-write and all the rolling over is taken care of by whatever policy you've defined in the index_policy lifecycle policy.

PUT index-write/_doc/1
{ ... }

As of 7.9, there's an easier way using data streams that doesn't need any bootstrapping like we did above. All you need to do is to create your index template like this:

PUT /_index_template/index_template
{
  "index_patterns": ["index"],                 
  "data_stream": {},
  "template": {
    "settings": {
      "number_of_shards": 1,
      "number_of_replicas": 1,
      "index.lifecycle.name": "index_policy"   
    }
  }
}

And then you can simply index your data like this

PUT index/_doc/1
{ ... }

Behind the scene, data stream indexes will be created and rolled over for you according to the index_policy lifecycle policy.

Upvotes: 1

Kaveh
Kaveh

Reputation: 1310

It is possible to have date-math expression in Rest API, for more information you can check ES offical documentation here https://www.elastic.co/guide/en/elasticsearch/reference/current/date-math-index-names.html.

For ILM policy you can find information here https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index-lifecycle-management.html

Upvotes: 0

Related Questions