CODEWITHSUNDEEP
elasticsearch
Akhadra
Akhadra

Reputation: 419

Why would I need to shrink index in elasticsearch

I tried searching for an answer to my question but I couldn't find any, this is my first time dealing with big data and Elasticsearch, I'm trying to learn how Elasticsearch works by going through there online tutorial, while reading I came across the topic for shrinking indices and how that can be done, OK now I know how to do it but unfortunately I don't know why I need to do it?

Why do I need to shrink my index and decrease my shards? is it space related change or what?

Upvotes: 2

Views: 4090

Answers (2)

xeraa
xeraa

Reputation: 10859

Every Elasticsearch index consists of multiple shards (default 5), which are each a Lucene index. Each one of these has an overhead (in terms of memory, file handles,...) but allow more parallelization. In case you don't need that much parallelization any more at some point — think of a daily index for logs and after a few days there won't be more writes any more and only few reads — you might want to reduce the number of shards to cut down on their overhead.

Upvotes: 5

nadre
nadre

Reputation: 517

The number of shards is tied to query performance in the following way:

How does shard size affect performance?

In Elasticsearch, each query is executed in a single thread per shard. Multiple shards can however be processed in parallel, as can multiple queries and aggregations against the same shard.

This means that the minimum query latency, when no caching is involved, will depend on the data, the type of query, as well as the size of the shard. Querying lots of small shards will make the processing per shard faster, but as many more tasks need to be queued up and processed in sequence, it is not necessarily going to be faster than querying a smaller number of larger shards. Having lots of small shards can also reduce the query throughput if there are multiple concurrent queries.

https://www.elastic.co/blog/how-many-shards-should-i-have-in-my-elasticsearch-cluster

Upvotes: 2

Related Questions