CODEWITHSUNDEEP
c++hashlarge-files
NSA
NSA

Reputation: 6027

How do I calculate the Hash of a large file with gcrypt, in c++

I have file that I need to hash, this file can be too large to read into memory at one time, so I need a way using gcrypt to hash the file in more manageable sized chunks how can I do this?

Thank you

Upvotes: 0

Views: 456

Answers (2)

NSA
NSA

Reputation: 6027

I found how to do this:

The gcrypt handle is valid and builds upon its input so that whether you write five 1k chunks to the handle or one 5k chunk of data the hash is the same either way. The hash itself is not finalized until you attempt to read it, so you can sit in a loop reading chunks of a large file at a time out and passing them to gcrypt and thus prevent yourself from ever having too much in memory at one time.

Upvotes: 1

Bruce Chidester
Bruce Chidester

Reputation: 621

I have not solved this problem before, but I can think of 2 possible solutions.

  1. Store a Hash Set. Lets say that your memory size is 1k and your file is 5k. You can have 5 pieces of this file. Take the hash of each piece and store the set of answers. In this example you would store 5 hash values to represent this file.

  2. Use option 1, then string the hash values together and take the hash of this.

Mostly it depends on your limits on how the hash values are stored.

Hope this helps.

Upvotes: 0

Related Questions