CODEWITHSUNDEEP
redis
Monster
Monster

Reputation: 15

Redis multi Key or multi Hash field

I have about 300k row data like this Session:Hist:[account]

Session:Hist:100000

Session:Hist:100001

Session:Hist:100002

.....

Each have 5-10 childs [session]:[time]

b31c2a43-e61b-493a-b8d4-ff0729fe89de:1846971068807

5552daa2-c9f6-4635-8a7c-6f027b4aa1a3:1846971065461

.....

I have 2 options:

  1. Using Hash, key is Session:Hist:[account], field is [session], value is [time]
  2. Using Hash flat all account, key is Session:Hist, field is [account]:[session], value is [time]

My Redis have 1 master, 4-5 Slave, using to store & push session (about 300k *5 in 2h) every days, and clear at end of day!

So the question is which options is better for performance (faster sync master-slave/smaller memory/faster for huge request), thanks for your help!

Upvotes: 1

Views: 3001

Answers (1)

Tasos P.
Tasos P.

Reputation: 4124

Comparing the two options mentioned, option #2 is less optimal.

According to official Redis documentation:

It is worth noting that small hashes (i.e., a few elements with small values) are encoded in special way in memory that make them very memory efficient.

More details here.

So having one huge hash with key Session:Hist would affect memory consumption. It would also affect clustering (sharding) since you would have one hash (hot-spot) located on one instance which would get hammered.

Option #1 does not suffer from the problems mentioned above. As long as you have many well-distributed (i.e. all accounts have similar count of sessions vs a few accounts being dominant with huge amount of sessions) hashes keyed as Session:Hist:[account].

If, however, there is a possibility for uneven distribution of sessions into accounts, you could try (and measure) the efficiency of option 1a:

  • Key: Session:Hist:[account]:[session - last two characters]
  • field: [session's last two characters]
  • value: [time]

Example:

  • Key: Session:Hist:100000:b31c2a43-e61b-493a-b8d4-ff0729fe89
  • field: de
  • value: 1846971068807

This way, each hash will only contain up to 256 fields (assuming last 2 characters of session are hex, all possible combinations would be 256). This would be optimal if redis.conf defines hash-max-zipmap-entries 256.

Obviously option 1a would require some modifications in your application but with proper bench-marking (i.e. memory savings) you could decide if it's worth the effort.

Upvotes: 3

Related Questions