CODEWITHSUNDEEP
c++
Ramesh
Ramesh

Reputation: 49

fclose() function slow

I tried to create around 4 GB file using c++ fopen, fwrite and fflush and fclose functions on Linux machine, but I observed that fclose() function is taking very long time to close the file, taking around (40-50 seconds). I checked different forum to find the reason for this slowness, changed the code as suggested in forums, Used setvbuf() function to make unbuffered I/O as like write() function. but still could not resolve the issue.

        totalBytes = 4294967296 // 4GB file
        buffersize = 2000;    
        while ( size <= totalBytes )
        {
            len = fwrite(buffer, 1, bufferSize, fp);
            if ( len != bufferSize ) {
                cout<<"ERROR (Internal): in calling ACE_OS::fwrite() "<<endl;
                ret = -1;
            }
            size = size + len;
        }
        ...
        ...
        ...
        fflush(fp);
        flcose(fp);

Any solution to the above problem would be very helpful.

thanks, Ramesh

Upvotes: 4

Views: 3208

Answers (4)

Mikko Juola
Mikko Juola

Reputation: 111

The operating system is deferring actual writing to the disk and may not actually write the data to the disk at any writing operation or even at fflush().

I looked at the man page of fflush() and saw the following note:

Note that fflush() only flushes the user space buffers provided by the C library. To ensure that the data is physically stored on disk the kernel buffers must be flushed too, for example, with sync(2) or fsync(2).

(there's a similar note for fclose() as well, although behaviour on your Linux system seems different)

Upvotes: 5

Torp
Torp

Reputation: 7924

Yeah, the time taken by fclose() is part of the time taken by the OS to write your data to the disk.
Look at fsync for achieving what you probably wanted with fflush. If you want to display some progress and the time used by fclose() is making it inaccurate, you could do a fsync() every 100 Mbytes written, or something like that.

Upvotes: 1

doug65536
doug65536

Reputation: 6781

fopen/fwrite/fclose are C standard wrappers around the low level open/write/close. All fflush is doing is making sure all the 'write' calls have been made for something buffered. There is no "synchronization point" at the fflush. The operating system is flushing the write buffer before it allows 'close' to return.

Upvotes: 1

Puppy
Puppy

Reputation: 146970

It will take a long time to write that much data to the disk, and there's no way around that fact.

Upvotes: 4

Related Questions