CODEWITHSUNDEEP
nosqlcompressionkey-value-store
David Tinker
David Tinker

Reputation: 9634

Distributed key value store with good on disk compression?

I need a simple key value store with good on disk compression and haven't had much luck Googling. It seems not too many people look at on disk size when evaluating stores.

I need to be able put/get persistent binary objects of between 1k and 100k in size with short string keys. Each key needs to be stored on at least 2 nodes for redundancy. Availability is more important than consistency. I don't need transactions, SQL queries or anything like that. The space on disk is the important thing for scaling my system.

Any suggestions?

Upvotes: 0

Views: 182

Answers (1)

David Tinker
David Tinker

Reputation: 9634

I ended up somewhat compressing my objects using Google protocol buffers and storing them in Apache Ignite with zstd compression turned on. This appears to be working well. The on disk size is about 80% of the original csv.gz source data.

Upvotes: 2

Related Questions