CODEWITHSUNDEEP
redisredis-cluster
badallen
badallen

Reputation: 1127

Optimizing Redis cluster nodes

I understand that in a Redis cluster, there are 16384 slots total distributed across the nodes. So if I have a key like this entity:user:userID (like user:1234) and the value is a serialized user object and say if my application has 500k+ users. It should get distributed to each slots evenly. We currently have 6 nodes total (3 masters and 3 slaves), and we are always wondering when we shall add 2 more nodes to 8 total. We also do write the cache data to disk, and sometimes we do get latency warning when persisting to disk. I'd assume if there are more nodes, there are less data to persist for each node, thus a better performance/usage of resources. But asides from disk i/o, is there a dead-on performance measurement to let us know when we should start adding additional nodes? Thanks!

Upvotes: 1

Views: 467

Answers (1)

Adam
Adam

Reputation: 2570

If your limiting factor is disk I/O for replication, using SSDs can drastically improve performance.

Two additional signs that it is time to scale out include server load and used memory for your nodes. There are others, but these two are simple to reason about.

If your limiting factor is processing power on the nodes (e.g. server load) because of a natural increase in requests, scaling out will help distribute the load across more nodes. If one node is consistently higher than the others, this could indicate a hot partition, which is a separate problem to solve.

If your limiting factor is total storage capacity (e.g. used memory) because of a natural increase in data stored in your cache, scaling out will help grow the total storage capacity of your cluster. If you have a very large dataset and the set of keys used on a regular basis is small, technologies such as Redis on Flash by Redis Labs may be applicable.

Upvotes: 1

Related Questions