CODEWITHSUNDEEP
elasticsearchelastic-stackttl
shiva
shiva

Reputation: 186

Elasticsearch: Auto delition with ILM doesn't work

I wanna delete an index after certain time(say 10s) but it doesn't work. I researched a lot but I couldn't find a different thing from my configs. Here are my configs:

my ILM config:

{
  "policy": {
    "phases": {
      "hot": {
        "min_age": "0ms",
        "actions": {
          "set_priority": {
            "priority": 100
          }
        }
      },
      "delete": {
        "min_age": "10s",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

my template:

{
  "order": 0,
  "index_patterns": ["myindex.*"],
  "settings": {
    "index": {
      "lifecycle": {
        "name": "myindex"
      }
    }
  }
}

my index:

{
  "job_id": 1,
  "message": "sample data",
  "@timestamp": "DATETIME"
}

then with the following script, I test my scenario:

#!/bin/bash                                                                                                                                                                            
NAME=myindex

# Add ILM and template.                                                                                                                                                                
echo -n "Add policy..."
http PUT http://localhost:9200/_ilm/policy/"${NAME}" < ilm.json                                                 
echo -n "Add template..."
http PUT http://localhost:9200/_template/"${NAME}" < template.json 

# Create index.                                                                                                                                                                        
echo -n "Create index..."
http PUT http://localhost:9200/"${NAME}" | jq '.acknowledged'

# List of data stream and indices.                                                                                                                                                     
echo -n "List of data stream..."
http http://localhost:9200/_data_stream/ | jq '.data_streams'
echo "List of indices..."
http http://localhost:9200/_cat/indices/ | awk '{print $3}'

echo "Index ILM explain..."
http http://localhost:9200/"${NAME}"/_ilm/explain

# Insert a document.                                                                                                                                                                   
echo -n "Insert a document..."
sed "s/DATETIME/$(date +%Y-%m-%dT%H:%M:%S)/g" < index.json | http POST http://localhost:9200/"${NAME}"/_doc | jq -r ".result"

# Wait until it reaches the TTL.                                                                                                                                                       
echo "Waiting to reach the TTL..."
sleep 10 # "$(( $(jq -r .policy.phases.delete.min_age < ilm.json | sed 's/s//g') + 2 ))"                                                                                               

# Search for the data and expected to find nothing.                                                                                                                                    
echo "Search the inserted document..."
echo '{"query": {"match": {"job_id": 1}}}' | http http://localhost:9200/"${NAME}"/_search

In the last part when I search for the index I can find it. The index is still present!!!

Upvotes: 1

Views: 422

Answers (1)

shiva
shiva

Reputation: 186

I set indices.lifecycle.poll_interval and it works!

PUT /_cluster/settings
{
    "persistent" : {
        "indices.lifecycle.poll_interval": "5s"
    }
}

References:

Upvotes: 1

Related Questions