CODEWITHSUNDEEP
elasticsearch
Joly
Joly

Reputation: 3276

How to limit existing index by size rather than number of days?

I'm trying to cap the size of the elastic search indexes by size so they don't fill up the disk space. If I understand correctly I can limit the size of each shard so that if limit is reached no new indexes are written in that shard however don't think I see an option to cap the size of an already created index? It can be capped by number of days but not size however as I can't predict how much data is going to be accumulated in X number of days I'd really rather give it a disk capacity size.

Is that possible?

Upvotes: 2

Views: 1355

Answers (1)

Eyal.Dahari
Eyal.Dahari

Reputation: 770

Indexes size is controlled also by the number of shards they are composed of. Shard size is a less reliable measure for index size because ongoing merges can produce significant temporary growth in shard size which disappears as soon as a merge is completed. Five primary shards, each in the process of merging to one 5GB shard, would temporarily increase the index size by 25GB! A better limitation would be the document count for each index which grows predictably or number of days as you mensioned.

Here is an article that explains in more details how to manage indexes. Please note that it focuses on time based indexes, but there are many insights which are related to your question.

Hope I have managed to help! :)

Upvotes: 1

Related Questions