CODEWITHSUNDEEP
linux
michael
michael

Reputation: 110670

Performance comparison in linux

I am trying to do performance comparison between 2 compression libraries on linux. They both take a file, do compression and then write to output file.

Since I am only interested in compression, I want to remove the file i/o overhead in my comparison.

Can I do this 

cat <source file> > my_compression_program /dev/null?

I just add timestamp in my comparison program in start and and. Since cat will read the file to memory, I should have no IO overhead.

Is my understanding correct?

Thank you.

Upvotes: 0

Views: 91

Answers (3)

amdn
amdn

Reputation: 11582

If you don't have enough memory to hold the file in memory, you have to do I/O.

If you do have plenty of memory then just run the benchmark twice, linux will keep the file in the page cache, see

Upvotes: 0

Peter - Reinstate Monica
Peter - Reinstate Monica

Reputation: 16112

You could use tmpfs (on *nix like OSs) if your libs really need a file handle. If not, your test program should just initialize some memory, possibly by reading a file, and compress that. In both cases you should consider turning off swapping for the test.

Upvotes: 0

phs
phs

Reputation: 11061

Nope!

cat does not buffer the entire file contents in memory before emitting any. Instead it reads and writes content in chunks determined by its internal buffer sizes.

If you wish to remove I/O disk overhead from your performance benchmark, I suggest you have plenty of ram and then cat the file to /dev/null before starting your benchmark:

cat source_file > /dev/null; my_compression_program < source_file > /dev/null

This will cause the file to first be inserted into the kernel's filesystem cache before your program ever runs. It will then be streamed out from memory.

Upvotes: 1

Related Questions