CODEWITHSUNDEEP
elasticsearchsharding
Matthieu Napoli
Matthieu Napoli

Reputation: 49573

How many shards should I use with Elasticsearch on a dev & CI environment?

By default Elasticsearch is configured to start with 5 shards.

Is there a reason to use 5 shards locally (on my development machine) and on the continuous integration server (for integration tests)? Is it better to use 1?

Obviously I don't care about scalability in those cases, I just want the simplest setup.

Upvotes: 3

Views: 230

Answers (2)

thibpat
thibpat

Reputation: 786

Once you've pushed some data with a specific shard configuration, you cannot set a different number of shards without re-index your data. So my guess is that the default configuration of elasticsearch is made so that you can scale your cluster to 5 nodes (then each node gets one shard) without headaches.

from the elasticsearch documentation:

A new index in Elasticsearch is allotted five primary shards by default. That means that we can spread that index out over a maximum of five nodes, with one shard on each node. That’s a lot of capacity, and it happens without you having to think about it at all!

Upvotes: 0

Andrei Stefan
Andrei Stefan

Reputation: 52368

The simplest setup is 1 primary shard, 0 replicas.

If you only have one node and replica count is >0 it will always be yellow. Not a problem per se, but those will not be needed.

If you want to test search response time with that one shard, for example, it depends on some factors if 1 is enough or you need more. The simplest rule of thumb is to have shards no larger than 30-50GB, for example. But this number also depends on factors.

So, I'd say if you have one node, start with 1 primary, 0 replicas. If that primary is too "large", think about having more primaries (each shard will do part of the work and each will use one core CPU for searching).

Upvotes: 2

Related Questions