CODEWITHSUNDEEP
cfile-iofopenfile-handlingfclose
duong_dajgja
duong_dajgja

Reputation: 4276

Should I call fopen - fclose every fwrite action

I am making something similar to commit log in database system. The system is able to handle ~ 20,000 events / sec. Each event occupies ~16 bytes. Roughly, the system will write to commit log at a speed of ~312.5 kB / sec. Each commit log file will contain at most of 500,000 events.

I have a question that: Should I call fopen - fwrite - fclose for each event, OR should I call fopen once when creating a new file, then a series of fwrite and finally fclose?

Upvotes: 1

Views: 1395

Answers (3)

tofro
tofro

Reputation: 6063

In such cases, it might be even better to revert to open/write/close and get rid of C buffered output completely. Log files are typically consisting of a high volume of nearly identical (size-wise) writes and do not really gain much from C buffering. Low level, unbuffered I/O would also relieve you of calling fflush() and can guarantee to write every single log entry as atomic entity.

Given the volume you mentioned, you should probably still not close and re-open the file between writes.

Upvotes: 3

autistic
autistic

Reputation: 15642

You're not obliged to, no... and in fact it would be a much better idea to call fflush as suggested by EvilTeach's answer.

However, better yet, if you can avoid calling fflush that would be ideal since the C standard library might (probably will) implement system-specific caching to unite smaller physical writes into larger physical writes, making your 20k writes per second more optimal.

Calling fopen/fwrite/fclose as you suggested, or fflush as EvilTeach suggested would elude that caching, which will probably degrade performance.

Upvotes: 0

EvilTeach
EvilTeach

Reputation: 28837

fopen/fwrite/fclose 20k times per second looks pretty expensive. Consider calling fflush as an alternative.

If you are looking to use it in order to record database transactions for possible recovery, you may need to rethink it. The f family of functions use buffering so in the event of a crash the final buffer may or may not have actually made it to disk.

Upvotes: 2

Related Questions