CODEWITHSUNDEEP
elasticsearchelasticsearch-curator
Goutham
Goutham

Reputation: 17

Elasticsearch Curator not to delete the last index while deleting indices by filtertypes

Using Elasticsearch 5.1 and Curator version is 4.3 in Centos 7

I am having some indices in elasticsearch whose naming format is sample.data.YYYY_MM_DD , sample.file.YYYY_MM_DD For example:-

sample.data.2019_07_22
sample.data.2019_07_23
sample.data.2019_07_25
sample.data.2019_07_26
sample.data.2019_07_28
sample.file.2019_07_21
sample.file.2019_07_25
sample.file.2019_07_26
sample.file.2019_07_29

I have used to run the action file by using the below command in Linux.

curator --config /root/config.yml /root/action_file.yml

I wanted to delete all indices except the recent index which is have created newer [ sample.data.2019_07_28, sample.file.2019_07_29 ]

This is the which i tried :-

  ---
  actions:
   1:
    action: delete_indices
    description: "Delete indices older than 3 days (based on index name), for workflow- prefixed indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly."
    filters:
      - 
        exclude: ~
        filtertype: pattern
        kind: prefix
        value: sample.*.
      - 
        direction: older
        exclude: ~
        filtertype: age
        source: name
        timestring: "%Y%m%d"
        unit: days
        unit_count: 3
    options:
      continue_if_exception: false
      disable_action: false
      ignore_empty_list: true
      timeout_override: ~

Its deleting overall indices even though i have used the below function also,

- filtertype: count
  count: 4

Expected output be like :-

sample.data.2019_07_28
sample.file.2019_07_29

Upvotes: 0

Views: 446

Answers (2)

untergeek
untergeek

Reputation: 863

I think you should upgrade to Curator 5.7, which fully supports Elasticsearch v5, and provides the count filter, which can sort indices by age and keep only n indices. Using the exclude flag, you can exclude the most recent index, and then use the regular age filter.

Upvotes: 0

Rob
Rob

Reputation: 9979

I think you should change your timestring from timestring: "%Y%m%d" to timestring: "%Y_%m_%d". When I test with a dry run I get:

2019-08-02 15:02:47,493 INFO      Preparing Action ID: 1, "delete_indices"
2019-08-02 15:02:47,513 INFO      Trying Action ID: 1, "delete_indices": Delete indices older than 3 days (based on index name), for workflow- prefixed indices. Ignore the error if the filter does not result in an actionable list of indices (ignore_empty_list) and exit cleanly.
2019-08-02 15:02:48,709 INFO      DRY-RUN MODE.  No changes will be made.
2019-08-02 15:02:48,709 INFO      (CLOSED) indices may be shown that may not be acted on by action "delete_indices".
2019-08-02 15:02:48,709 INFO      DRY-RUN: delete_indices: sample.file.2019_07_26 with arguments: {}
2019-08-02 15:02:48,709 INFO      DRY-RUN: delete_indices: sample.file.2019_07_27 with arguments: {}
2019-08-02 15:02:48,710 INFO      DRY-RUN: delete_indices: sample.file.2019_07_28 with arguments: {}
2019-08-02 15:02:48,710 INFO      DRY-RUN: delete_indices: sample.file.2019_07_29 with arguments: {}
2019-08-02 15:02:48,710 INFO      DRY-RUN: delete_indices: sample.file.2019_07_30 with arguments: {}
2019-08-02 15:02:48,710 INFO      Action ID: 1, "delete_indices" completed.
2019-08-02 15:02:48,710 INFO      Job completed.

Hope that helps.

Upvotes: 1

Related Questions