CODEWITHSUNDEEP
elasticsearchlogstash
SolarCore
SolarCore

Reputation: 133

How to remove documents from elasticsearch by date period or by size?

I want to remove data from elasticsearch with configuring the data date period, for example, data older that 30 days, or maybe by size, like if index size is bigger than 100Mb then remove old data. I use logstash to move logs to one index in elasticsearch. How can i do that?

Upvotes: 5

Views: 8795

Answers (2)

Kaushik J
Kaushik J

Reputation: 1062

If you want to explicitly delete logs based on ur criteria you can use delete_by_query

POST /my_logs/_delete_by_query
{
    "query": {
        "range": {
            "date": {
                "lte": <your_target_date>
            }
        }
    }
}

Upvotes: 7

Tanimak
Tanimak

Reputation: 184

Deleting older data from the current index is not a easy way to do it. You can configure logstash to create new index daily. Then you can access all your data through index patterns or alias.

Then you will be able to delete older indexes without much issue based on the date.

You can automate these using curator - https://www.elastic.co/guide/en/elasticsearch/client/curator/5.8/index.html

See this post on configuring logstash to create indexes daily Create a new index per day for Elasticsearch in Logstash configuration

Upvotes: 2

Related Questions