CODEWITHSUNDEEP
cfilefclosefflush
piet_lu
piet_lu

Reputation: 97

why fclose() is not always flushing the data to the disk?

I'm getting some weird results, while trying to write data to files in C.<br/> I thought that fclose() closes the *FILE and flushes the data from its buffer to the file.<br/> But for some reason it only flushes the data in my program sometimes and it doesn't do it other times.

For example: I would run this code, and in my file I can see the two strings. (Perfect, exactly what I want)
But then when I run the code the next 4 times it doesn't change anything in my file. And then when I run it another time, suddenly the 10 extra strings appear (8 from the last times I ran the program and the 2 from now)
(This 4 times is just an example, sometimes it's 5, 8, 10, or even just 2 times before I see the output appear)

I really don't understand this? Shouldn't the data be visible after every time I run the program? Where is this buffer even saved between the different times I run the program, because the program finishes every time, so the memory gets released, right?

(By the way I also tried fflush(fd) before and after fclose(), but that didn't solve the problem)

#include <stdio.h>


int main(int argc, char const *argv[]) {
  FILE * fd;
  fd = fopen("test_file.txt", "a");
  fprintf(fd, "String 1\n");
  fprintf(fd, "String 2\n");
  fclose(fd);
  return 0;
}

Upvotes: 5

Views: 3900

Answers (1)

AdamF
AdamF

Reputation: 2940

Thanks to michael kerrisk we have all the answers we need in linux man pages. Will if you dig in the man pages and see the Notes section (listed below in my answer) you will understand this behavior.

int fclose(FILE *stream); - close a file stream pointed by stream.

DESCRIPTION

The fclose() function flushes the stream pointed to by stream (writing any buffered output data using fflush()) and closes the underlying file descriptor.

NOTES

Note that fclose() flushes only the user-space buffers provided by the C library. To ensure that the data is physically stored on disk the kernel buffers must be flushed too, for example, with sync() or fsync().

So its not enough to ensure writing and flushing your buffers, we need to flush the kernel buffers as well. how ? see below: make sure that you are reading my comments in the code as well!

#include <unistd.h>
#include <stdio.h>

int main(int argc, char const *argv[]) {
  FILE * fd;
  fd = fopen("test_file.txt", "a");
  fprintf(fd, "String 1\n");
  fprintf(fd, "String 2\n");
  
  if(fclose(fd) != 0){
/*********************************************************************
fclose failed and errno will be set to endicate the error. be aware 
that we shall 
not call fsync or any other stream manipulation on fd in this case 
because we will get undefined behavior !!!
**********************************************************************/
      ... do work but no more work on this stream (fd)...
  }

  /* ~~ be aware that this is a blocking call ! (see details below) ~~*/
  if(fsync(fd) == -1){
      /*fsync fails*/
      ....
  } 
  return 0;
}

fsync() transfers ("flushes") all modified in-core data of (i.e., modified buffer cache pages for) the file referred to by the file descriptor fd to the disk device (or other permanent storage device) so that all changed information can be retrieved even if the system crashes or is rebooted. This includes writing through or flushing a disk cache if present. The call blocks until the device reports that the transfer has completed

One more important thing, always prefer calling fsync(fd) rather than calling void sync(void); ! why ?

because sync() causes all pending modifications to filesystem metadata and cached file data to be written to the underlying filesystems. usualy we dont want this kind of behavior (and this trigger the kernel to do extra work that is not essential! so dont call it.

this page and answer is not large enough to include all the details and special cases and all error codes! please refer to man pages and to the links belwo:

fclose

fsync

Upvotes: 6

Related Questions