CODEWITHSUNDEEP
hadoopcompressionhbasesnappy
pacman
pacman

Reputation: 835

Hbase: How does reading of compressed data work?

I understand that if we enable compression for column family then the values of hfiles saved will be compressed using the algorithm we specify and compression happens at block level.

But when reading the data block out how does this decompression work. For eg. If we use snappy how does the read happen. How does it figure out how to read a specific key value from the compressed data? How is this done in a real time basis.

Upvotes: 0

Views: 163

Answers (1)

Matt Andruff
Matt Andruff

Reputation: 5155

The data is decompressed in memory 'on the fly' and the value is then read in uncompressed format. This is done quickly as using cpu to uncompress is faster than doing large reads from file.

The decompression happens while the file is being read from disk.

Upvotes: 1

Related Questions