CODEWITHSUNDEEP
c++windowsfile-iocreatefilewritefile
Michael Kelley
Michael Kelley

Reputation: 3689

saving file to network

I have a file that's about 7 MB that saves to my local share in a matter of seconds. However, saving that file to a network location takes minutes. I'm wondering what I can do to speed this up. Here are my current options:

  1. Save the data to a temporary file on the local machine, then copy the temporary file over to the network path. I'll probably do this as this is the easiest and the most bang for the buck.
  2. Use SetFilePointerEx() and SetEndOfFile(). I thought this might be useful based on the answer to this question: Creating big file on Windows
  3. Buffer writes. I could cache write data myself and flush when the buffer full, but wouldn't this be redundant with the caching that is already done by the OS?

#1 seems like the best option, but I'm wondering if anyone has any advice on a better way to speed up saving to network paths?

Edit: The network is on a gigabit LAN, so speed shouldn't be an issue. Copying the file to the network path takes about 1 second. I just noticed we're calling WriteFile() on smaller chunks of data then we probably should, so optimizing the higher level code to write bigger chunks will probably help, but the speed difference is still so significant that it's still a worthwhile question to ask.

Upvotes: 1

Views: 978

Answers (3)

MSalters
MSalters

Reputation: 180295

You'll want to aovid read-modify-write operations. You'll typically want to write blocks of at least 4KB, possibly higher powers of 2. The reason is that to append one byte, you usually need to read the last block of a file, append one byte, and write back the new block. By writing 4KB blocks (only), every write typically ends up as a new block at the end of the file.

Caching should help you here, but caching isn't perfect. It may help to open the file exclusively. If you deny read access, the OS might notice that flushing the cache isn't too important for other apps.

CopyFile can be fast because it can do exactly the same.

Upvotes: 1

ChrisW
ChrisW

Reputation: 56123

I'm wondering if anyone has any advice on a better way to speed up saving to network paths?

Maybe you need a better network. ISPs often provide fast downloads but slow uploads. How long does it take to transfer 7 MB using a protocol such as FTP?

Upvotes: 1

Aaron M
Aaron M

Reputation: 2563

Are you running on a slow network?

Id go with option number 1 and save the file to the network share in the background

Upvotes: 0

Related Questions