CODEWITHSUNDEEP
redis
Mike Ellery
Mike Ellery

Reputation: 2094

Is there a practical limit to the number of elements in a sorted set in redis?

I'm currently migrating some data to Redis and I'm considering using a sorted set to store approximately 1.4e6 items (with associated scores/counts). Is this number of items in a set likely to exceed a practical limit, making it too painful to use the set? I plan on running 64 bit redis, so available memory for the data should not be a problem. Does anyone have experience with a sorted set this size? If so, how are your insertion and query times for the set?

Upvotes: 30

Views: 18640

Answers (2)

Kien Nguyen
Kien Nguyen

Reputation: 408

Our site has a sorted set with about 2 milions items (email addresses) with integer scores and it took up about 320MB in memory size.

Upvotes: 19

Tom Clarkson
Tom Clarkson

Reputation: 16174

It depends what you want to do with the set. The simple operations are mostly O(log n) which means that they take only twice as long for a million item set as they do for a thousand item set. Unless you have something seriously broken in your config like a memory limit smaller than the set, performance shouldn't be a problem.

Where you need to be careful is with operations on multiple sets, particularly union - that will take a thousand times longer for the million item set. In practical terms this isn't necessarily a problem though - either it will be fast enough for your purposes anyway (redis has commands documented as too slow for production use that are still best measured in milliseconds) or you can adjust the order of operations to avoid running union on really large sets.

Upvotes: 27

Related Questions