CODEWITHSUNDEEP
redisset
Pirooz
Pirooz

Reputation: 1278

How to efficiently union non-overlapping sets in redis?

I have a use case where I know for a fact that some sets I have materialized in my redis store are disjoint. Some of my sets are quite large, as a result, their sunion or sunionstore takes quite a large amount of time. Does redis provide any functionality for handling such unions?

Alternatively, if there is a way to add elements to a set in Redis without checking for uniqueness before each insert, it could solve my issue.

Upvotes: 4

Views: 1418

Answers (1)

Didier Spezia
Didier Spezia

Reputation: 73216

Actually, there is no need for such feature, because of the relative cost of operations.

When you build Redis objects (such as sets or lists), the cost is not dominated by the data structure management (hash table or linked lists), because the amortized complexity of individual insertion operations is O(1). The cost is dominated by the allocation and initialization of all the items (i.e. the set objects or the list objects). When you retrieve those objects, the cost is dominated by the allocation and formatting of the output buffer, not by the access paths in the data structure.

So bypassing the uniqueness property of the sets does not bring a significant optimization.

To optimize a SUNION command if the sets are disjoint, the best is to replace it by a pipeline of several SMEMBERS commands to retrieve the individual sets (and build the union on client side).

Optimizing a SUNIONSTORE is not really possible since disjoint sets is a worst case for the performance. The performance is dominated by the number of resulting items, so the less items in common, the more response time.

Upvotes: 5

Related Questions