CODEWITHSUNDEEP
cachingredisin-memory-databasein-memory
Tibin Paul
Tibin Paul

Reputation: 846

Is there a limit in number of keys in a Redis SET UNION operation?

I have a scenario where I am dumping huge amount of data from Google Big Query to Redis SET data structure to get better response time. I need SET UNION operation to be done over millions of keys. I have tested with few thousands of keys and working fine. The question is, there is any limit on number of keys can be supplied to a SUNION command at a time? Is it really SUNION Key1 Key2 Key3 ..... KeyN? Consider I have enough system capacity.

Upvotes: 2

Views: 1564

Answers (1)

Matías Fidemraizer
Matías Fidemraizer

Reputation: 64943

[...] over millions of keys

There's no statement in Redis' documentation about a limitation on how many keys can be provided in a single sunion command.

BTW, I doubt that doing such operation could be a good idea in Redis. Remember that Redis will get blocked until this operation and, and no other operation will be executed until the sunion ends.

My best advise will be you should do it using many sunionstore commands, and later get all results from many sets like if the whole sets would be pages of the result of sunion millions of keys.

sunionstore key:pages:1 key1 keyN

...and later you would use some iterator in your application layer to iterate over all generated pages.

Upvotes: 1

Related Questions