CODEWITHSUNDEEP
elasticsearchelasticsearch-6.5
jbmyid
jbmyid

Reputation: 2015

Elasticsearch maximum index count limit

Is there any limit on how many indexes we can create in elastic search?

Can 100 000 indexes be created in Elasticsearch?

I have read that, maximum of 600-1000 indices can be created. Can it be scaled?

eg: I have a number of stores, and the store has items. Each store will have its own index where its items will be indexed.

Upvotes: 3

Views: 10186

Answers (1)

Amit
Amit

Reputation: 32376

There is no limit as such, but obviously, you don't want to create too many indices(too many depends on your cluster, nodes, size of indices etc), but in general, it's not advisable as it can have a server impact on cluster functioning and performance.

Please check loggly's blog and their first point is about proper provisioning and below is important relevant text from the same blog.

ES makes it very easy to create a lot of indices and lots and lots of shards, but it’s important to understand that each index and shard comes at a cost. If you have too many indices or shards, the management load alone can degrade your ES cluster performance, potentially to the point of making it unusable. We’re focusing on management load here, but running too many indices/shards can also have pretty significant impacts on your indexing and search performance.

The biggest factor we’ve found to impact management overhead is the size of the Cluster State, which contains all of the mappings for every index in the cluster. At one point, we had a single cluster with a Cluster State size of over 900MB! The cluster was alive but not usable.

Edit: Thanks @Silas, who pointed that from ES 2.X, cluster state updates are not that much costly(As the only diff is sent in update call). More info on this change can be found on this ES issue

Upvotes: 4

Related Questions