CODEWITHSUNDEEP
clinux
Victor
Victor

Reputation:

Why the fwrite libc function is faster than the syscall write function?

After providing the same program which reads a random generated input file and echoes the same string it read to an output. The only difference is that on one side I'm providing the read and write methods from linux syscalls, and on the other side I'm using fread/fwrite.

Timing my application with an input of 10Mb in size and echoing it to /dev/null, and making sure the file is not cached, I've found that libc's fwrite is faster by a LARGE scale when using very small buffers (1 byte in case).

Here is my output from time, using fwrite:

real    0m0.948s
user    0m0.780s
sys     0m0.012s

And using the syscall write:

real    0m8.607s
user    0m0.972s
sys     0m7.624s

The only possibility that I can think of is that internally libc is already buffering my input... Unfortunately I couldn't find that much information around the web, so maybe the gurus here could help me out.

Upvotes: 26

Views: 23894

Answers (3)

Matthew Iselin
Matthew Iselin

Reputation: 10660

Timing my application with an input of 10Mb in size and echoing it to /dev/null, and making sure the file in not cached, I've found that libc's frwite is faster by a LARGE scale when using very small buffers (1 byte in case).

fwrite works on streams, which are buffered. Therefore many small buffers will be faster because it won't run a costly system call until the buffer fills up (or you flush it or close the stream). On the other hand, small buffers being sent to write will run a costly system call for each buffer - that's where you're losing the speed. With a 1024 byte stream buffer, and writing 1 byte buffers, you're looking at 1024 write calls for each kilobyte, rather than 1024 fwrite calls turning into one write - see the difference?

For big buffers the difference will be small, because there will be less buffering, and therefore a more consistent number of system calls between fwrite and write.

In other words, fwrite(3) is just a library routine that collects up output into chunks, and then calls write(2). Now, write(2), is a system call which traps into the kernel. That's where the I/O actually happens. There is some overhead for simply calling into the kernel, and then there is the time it takes to actually write something. If you use large buffers, you will find that write(2) is faster because it eventually has to be called anyway, and if you are writing one or more times per fwrite then the fwrite buffering overhead is just that: more overhead.

If you want to read more about it, you can have a look at this document, which explains standard I/O streams.

Upvotes: 40

DigitalRoss
DigitalRoss

Reputation: 146053

write(2) is the fundamental kernel operation.

fwrite(3) is a library function that adds buffering on top of write(2).

For small (e.g., line-at-a-time) byte counts, fwrite(3) is faster, because of the overhead for just doing a kernel call.

For large (block I/O) byte counts, write(2) is faster, because it doesn't bother with buffering and you have to call the kernel in both cases.

If you look at the source to cp(1), you won't see any buffering.

Finally, there is one last consideration: ISO C vs Posix. The buffered library functions like fwrite are specified in ISO C whereas kernel calls like write are Posix. While many systems claim Posix-compatibility, especially when trying to qualify for government contracts, in practice it's specific to Unix-like systems. So, the buffered ops are more portable. As a result, a Linux cp will certainly use write but a C program that has to work cross-platform may have to use fwrite.

Upvotes: 15

dmityugov
dmityugov

Reputation: 4478

You can also disable buffering with setbuf() function. When the buffering is disabled, fwrite() will be as slow as write() if not slower.

More information on this subject can be found there: http://www.gnu.org/s/libc/manual/html_node/Controlling-Buffering.html

Upvotes: 11

Related Questions