What is the maximum number of indices allowed in elasticsearch?

Question

I am using MySQL for storing data in my software. We have to generate hundreds of reports using the data and it is taking more time when the size of data in increasing. For some reports it is taking 1 or 2 minutes. So we have decided to use NoSQL database instead of MySQL. We have decided to go with Elasticsearch.

In MySQL, we have different database for each client and inside each database there are more than 50 tables. When coming to Elasticsearch, I am not able to group indices under a database. So I have decided to create separate index with placing client name as prefix. For example in MySQL there are 2 databases named ABC, DEF and a table inside both named ledger. The same thing done in Elasticsearch is by creating an index with name ABC_ledger and DEF_ledger.

In MySQL a single client database has 50 tables so when converting to Elasticsearch it will be 50 index for a single client. So when new clients are added, an additional 50 indices will be created.

My questions are:

1. Is there any limit in number of indices we can create?
2. Is there any other way to implement above logic without creating separate indices?

Answer 1

I'll try to answer your first question.

1. First, Indices in Elasticsearch is just the logical grouping of the primary shards and replica shards(these shards are actually allocated on the nodes of a Elasticsearch cluster). There are two configs, which controls how many shards can be inside a cluster or data nodes.

First one called cluster.max_shards_per_node is applied at a cluster level, that limits the total number of shards in the cluster irrespective of nodes in the cluster, default is set to 1000. Read more at cluster module, and from same

2. First one isn't enough as it doesn't limit the no of shards at node level but limits that to cluster level that can create the imbalance in the cluster, to prevent that you can set number of shards at the node level. read more at official document