CODEWITHSUNDEEP
linuxhashparallel-processingmd5sha
Avio
Avio

Reputation: 2708

Applying different hashing algorithms to the same file at the same time

Is it possible to apply different hashing algorithms (MD4, MD5, SHA1, SHA256, SHA512, etc.) to the same file at the same time under Linux?

This is especially useful when processing large files to avoid reading from disk the same content multiple times, but it should be also very useful to distribute work on multi-core processors.

I suspect that I need something like tee but I can't figure out an easy solution to this problem.

EDIT:

Thanks to tink's answer, this is exactly what I was looking for:

#> time ( cat disk.img | tee >( md5sum > md5 ) | tee >( sha1sum > sha1 ) | tee >( sha256sum > sha256 ) | tee >( sha512sum > sha512 ) > /dev/null )

real    1m2.801s
user    0m1.272s
sys     0m18.505s

And this was the slow sequential method:

#> time ( md5sum disk.img && sha1sum disk.img && sha256sum disk.img && sha512sum disk.img )
34f3b8bc1b27777a31b7d46363062ae3  disk.img
85bed81808d6fe4c0ade68595d0f16b008cca57b  disk.img
255308c8887759479fe63b8bc93981001e909f7198593a023ccb0d8986a3a6ea  disk.img
86a2af98bdb9dfefbe54ecd941de614b773218e50dc9eea4ea8d79b443f3c1af50657085dcdbfd161e3f1ec3e91b2f9f7d5859b55f3aee44a7d554f1854e7890  disk.img

real    3m29.099s
user    1m53.459s
sys     0m10.573s

Upvotes: 3

Views: 507

Answers (1)

tink
tink

Reputation: 15204

Something like this should work:

cat iso-image  | tee >( md5sum  > md5 2>&1 ) |( sha256sum > sha256 2>&1)

Insert extra tee > ( method > methodfile 2>&1 ) blocks for the others before the sha256 at the end.

Upvotes: 4

Related Questions