CODEWITHSUNDEEP
c++clinuxlinux-kernelmemory-mapped-files
intrigued_66
intrigued_66

Reputation: 17278

Fastest way of reading a file in Linux?

On Linux what would be the fastest way of reading a file in to an array of bytes/to process the bytes? This can include memory-mapping, sys calls etc. I am not familiar with the many Linux-specific functions.

In the past I have used boost memory mapping, but I need faster Linux-specific performance rather than portability.

Upvotes: 4

Views: 4159

Answers (2)

Art
Art

Reputation: 20402

mmap should be the fastest way to access the contents of a file if the file is large enough. There's an initial cost for setting up the memory mappings, but that's offset by not needing to copy the data from the page cache into userland. And if you want all the contents of the file, the cost to allocate the memory to your program should be more or less the same as the cost of mmap.

Your best bet, as always, is to test and benchmark.

Upvotes: 5

Alfe
Alfe

Reputation: 59606

Don't let yourself get fooled by lazy stuff like memory mapping. Rather focus on what you really need. Do you really need to read the whole file into memory? Then the straight-forward way of opening, reading chunks in a loop, and closing the file will be as fast as it can be done.

But often you don't really want that. Instead you might want to read specific parts, a block here, a block there, jump through the file, read a block at a specific position, etc.

Then still fseeking out those positions and freading the blocks won't have overheads worth mentioning. But it can be more convenient to use memory mapping to let the operating system or a library deal with stuff like memory allocation etc. It won't get the job done faster, though.

Upvotes: 3

Related Questions