CODEWITHSUNDEEP
c++stlvector
Pedro d'Aquino
Pedro d'Aquino

Reputation: 5220

Efficient way of reading a file into an std::vector<char>?

I'd like to avoid unnecessary copies. I'm aiming for something along the lines of:

std::ifstream testFile( "testfile", "rb" );
std::vector<char> fileContents;
int fileSize = getFileSize( testFile );
fileContents.reserve( fileSize );
testFile.read( &fileContents[0], fileSize );

(which doesn't work because reserve doesn't actually insert anything into the vector, so I can't access [0]).

Of course, std::vector<char> fileContents(fileSize) works, but there is an overhead of initializing all elements (fileSize can be rather big). Same for resize().

This question is not so much about how important that overhead would be. Rather, I'm just curious to know if there's another way.

Upvotes: 40

Views: 33904

Answers (3)

wilhelmtell
wilhelmtell

Reputation: 58667

The canonical form is this:

#include<iterator>
// ...

std::ifstream testFile("testfile", std::ios::binary);
std::vector<char> fileContents((std::istreambuf_iterator<char>(testFile)),
                               std::istreambuf_iterator<char>());

If you are worried about reallocations then reserve space in the vector:

#include<iterator>
// ...

std::ifstream testFile("testfile", std::ios::binary);
std::vector<char> fileContents;
fileContents.reserve(fileSize);
fileContents.assign(std::istreambuf_iterator<char>(testFile),
                    std::istreambuf_iterator<char>());

Upvotes: 66

Maxim Egorushkin
Maxim Egorushkin

Reputation: 136208

If you want true zero-copy reading, that is, to eliminate copying from kernel to user space, just map the file into memory. Write your own mapped file wrapper or use one from boost::interprocess.

Upvotes: 5

roxrook
roxrook

Reputation: 13843

If I understand you correctly, you want to read each element but don't want to load it all into the fileContents, correct? I personally don't think this would make unnecessary copies because open files multiple times would decrease performance more. Read once into a fileContentsvector is a reasonable solution in this case.

Upvotes: 0

Related Questions