CODEWITHSUNDEEP
c#.netelasticsearchnest
user4856537
user4856537

Reputation:

How to set a TTL for an index in ElasticSearch

I would like to know how can I set a TTL so that each document older than 30 days are automatically removed by the ES cluster.

I know previous versions used to manage this using the TTL field but it seems it is removed now.

I'm using NEST.NET 6.4.0

Upvotes: 2

Views: 1859

Answers (2)

Bui Hai Duong
Bui Hai Duong

Reputation: 271

Let try this way.

First, you create a policy to describe when an index will be deleted

PUT http://localhost:9200/_ilm/policy/delete_log_after_2day <- Your policy name

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 0
          }
        }
      },
      "delete": {
        "min_age": "2d", <-- Set your TTL here
        "actions": {
          "delete": {
            "delete_searchable_snapshot": true
          }
        }
      }
    }
  }
}

Next, create a template to select which type of index will use this policy.

PUT http://localhost:9200/_index_template/delete_after2day_template

{
"index_patterns": [
    "test*" <-- Choose your index here
],
"template": {
    "settings": {
        "number_of_shards": 1,
        "number_of_replicas": 1,
        "index.lifecycle.name": "delete_log_after_2day"
    }
}
}

Now, when you create a new index example: test001 it will be automatically deleted after 2 days.

Note: for the old index, it will not assign to your new policy, so it won't be deleted except you assign it.

You can use this API to assign all old indexes to your policy

PUT http://localhost:9200/test*/_settings <--Your old index here, can use pattern

{
  "index": {
    "lifecycle": {
      "name": "delete_log_after_2day" <-- Your policy name
    }
  }
}

And then after your expiration date, all old indexes will be deleted completely.

Upvotes: 2

Ijaz Ahmad
Ijaz Ahmad

Reputation: 12110

TTL was removed in 5.0

you can use curator , shedule it as cronjob or jenkins job , and delete old indices all together.

curator_cli --host  localhost --port 9200 --http_auth "$USER":"$PASS" \
delete_indices --ignore_empty_list --filter_list '[{"filtertype":"age","source":"creation_date","direction":"older","unit":"days","unit_count":30}, \
{"filtertype":"pattern","kind":"prefix","value":"livelogs"}]'

See also:

Is there a way to Set ttl in elastic search index

Upvotes: 1

Related Questions