CODEWITHSUNDEEP
elasticsearch-curator
Martin
Martin

Reputation: 2581

ElasticSearch Curator delete unless last entry

Is there a way to stop curator deleting the last index when deleting by time?

actions:
1:
  action: delete_indices
  description: Delete kube- indices older than 14 days. Ignore the error if there are none and exit cleanly.
  options:
    disable_action: False
    ignore_empty_list: True      
  filters:
  - filtertype: pattern
    kind: prefix
    value: kube-
  - filtertype: age
    source: name
    direction: older
    timestring: '%Y.%m.%d'
    unit: days
    unit_count: 14

I have this which works great to keep a currently running K8s cluster logs in check. However when we move AWS region the log name changes e.g. from kube-eu-west-1-<date> to kube-eu-west-2-<date>.

Curator diligently cleans up all the data after 14 days. What I'd like is to prevent it from removing the last entry for a particular index, so there is always a record of what happened the last time the cluster was in that region.

(It would also "fix" some less well written pieces of code that throw errors when the data they expect to be there has legitimately gone away).

Upvotes: 0

Views: 270

Answers (1)

untergeek
untergeek

Reputation: 863

You could use the count filter:

filters:
  # your existing filters go BEFORE the count filter...
  - filtertype: count
    count: 1

This example should exclude (in spite of the exclude: false) the most recent index from the list of actionable indices, preserving it. If this is not the index you want to exclude, explore using exclude: true/false (default is true), and/or reverse: true/false (default is true) until it excludes the index you want.

NOTE: Always use the --dry-run flag to test your filters before deploying them on actual data. Iterate until it looks right. Use of loglevel: DEBUG in your client settings will show how filters make their decisions, if that helps.

Upvotes: 1

Related Questions