CODEWITHSUNDEEP
elasticsearchelasticsearch-curator
Rexave
Rexave

Reputation: 435

Elasticsearch curator delete the whole index instead of older documents

I have an index which contains data for 1 year. I want to delete the older data and keep the last 60 days.

I tried with delete_query which seems to work but it's VERY time consuming.

It seems "curator" is the best way to do this.

So, I made this configuration :

actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 60 days.
    options:
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: myindex*
    - filtertype: age
      source: field_stats
      direction: older
      unit: days
      unit_count: 60
      field: '@timestamp'
      stats_result: min_value

and launch it with curator cli.

Versions : curator 5.8 and elasticsearch 6.5

The result is not the one expected !

If there is data older than 60 days, the WHOLE index is dropped instead of only the data older than 60 days. Can you help me with that please ?

Upvotes: 0

Views: 716

Answers (1)

untergeek
untergeek

Reputation: 863

Curator only deletes entire indices. This has always been the case from the beginning. You have two options: stick with delete by query, or split your data into multiple indices to expire it over time in chunks (entire indices). Because rollover indices are backed by a single alias, your searches will still see the entire data set (when searching using the alias).

Upvotes: 1

Related Questions