CODEWITHSUNDEEP
c++fileread-write
Lion King
Lion King

Reputation: 33813

What is the secret to the speed of the filesystem::copy function?

I am trying to reach the speed of filesystem::copy in reading the content of a file and write that content to a new file "copy operation" but I can't reach that speed.

The following is a simple example of my attempt:

void Copy(const wstring &fromPath, const wstring &toPath) {
    ifstream readFile(fromPath.c_str(), ios_base::binary|ios_base::ate);    
    char* fileContent = NULL;
    if (!readFile) { cout << "Cannot open the file.\n"; return; }
    ofstream writeFile(toPath.c_str(), ios_base::binary);
    streampos size = readFile.tellg();
    readFile.seekg(0, ios_base::beg);
    fileContent = new char[size];
    readFile.read(fileContent, size);
    writeFile.write(fileContent, size);
    readFile.close();
    writeFile.close();
    delete[] fileContent;
}

The previous code able to copy a file.iso its size "1.48GB" in between "8 to 9" seconds, while filesystem::copy able to copy the same file in between "1 to 2" seconds maximum.
Notice: I don't want to use C++17 in the current period.

How can I do to make the speed of my function to be like filesystem::copy?

Upvotes: 1

Views: 298

Answers (1)

Jeffrey
Jeffrey

Reputation: 11410

Your implementation needs to allocate a buffer of the size of the whole file. That is wasteful, you could just read 64k, write 64k, repeat for the next blocks.

There's cost to paging memory in and out. If you read the whole thing then write the whole thing, you end up paging in and out the whole file twice.

It could be that multiple threads might read/write separately (provided read stays ahead). That may speed things up.

With hardware support, there might not even be a need for the data to go all the way to the CPU. Yet, your implementation probably ends up doing it. It would be very hard hard for the compiler to reason about what you do or don't with fileContent.

There's countless other tricks the implementation of filesystem::copy might be using. You could go see how it is coded, there's plenty of open implementations.

There's a caveat though: The implementation of the standard library might rely on specific behaviours that the language doesn't guarantee. So you can't simply copy the code to a different compiler/architecture/platform.

Upvotes: 4

Related Questions