CODEWITHSUNDEEP
c++io
Puddle
Puddle

Reputation: 3345

fstream faster than fopen? and gets faster as buffers increase?

I've been trying to write file data as fast as possible.

For some reason fstream is faster than fopen.

I've been hearing that C's file I/O is faster (which makes sense)
<fstream> includes <stdio.h> yet i can't get fopen to perform as fast.

NOTE (old questions):

also realized fstream.put(char) is faster than fstream << char
(otherwise fopen is faster than fstream if the buffer is < ~256)

Here's my testing:

#include <iostream>
#include <fstream>
#include <ctime>

int filesize; // total bytes (individually "put" in buffered stream)
int buffsize; // buffer size

void writeCPP(){
    std::ofstream file;
    char buffer[buffsize]; file.rdbuf()->pubsetbuf(buffer,buffsize);    // set buffer (before opening)
    file.open("test.txt",std::ios::binary);                             // open file
    for(int i=0; i<filesize; i++) file.put('a');                        // write bytes
    file.close();                                                       // close
}

void writeC(){
    FILE* file=fopen("test.txt","wb");                                  // open file
    char buffer[buffsize]; setvbuf(file,buffer,_IOFBF,buffsize);        // set buffer
    for(int i=0; i<filesize; i++) fputc('a',file);                      // write bytes
    fclose(file);                                                       // close
}

#define getTime() double(clock())/CLOCKS_PER_SEC // good enough

double start;

void test(int s){ // C++ vs C (same filesize / buffsize)
    buffsize=s;
    std::cout<<"  buffer: "<<buffsize<<"\t"<<std::flush;

    start=getTime();
    writeCPP();
    std::cout<<"  C++: "<<getTime()-start<<",\t"<<std::flush;

    start=getTime();
    writeC();
    std::cout<<" C: "<<getTime()-start<<std::endl;
}

#define MB (1024*1024)

int main(){
    filesize=10*MB;
    std::cout<<"size: 10 MB"<<std::endl;

    // C++ fstream faster
    test(64);   // C++ 0.86 < C 1.11 (1.29x faster)
    test(128);  // C++ 0.44 < C 0.79 (1.80x faster) (+0.51x)
    test(256);  // C++ 0.27 < C 0.63 (2.33x faster) (+0.53x)
    test(512);  // C++ 0.19 < C 0.56 (2.94x faster) (+0.61x)
    test(1024); // C++ 0.15 < C 0.52 (3.46x faster) (+0.52x)
    test(2048); // C++ 0.14 < C 0.51 (3.64x faster) (+0.18x)
    test(4096); // C++ 0.12 < C 0.49 (4.08x faster) (+0.44x)
    test(8192); // C++ 0.10 < C 0.48 (4.80x faster) (+0.72x)
}

Upvotes: 1

Views: 3659

Answers (3)

Miles Budnek
Miles Budnek

Reputation: 30639

The only standard-defined behavior for std::basic_filebuf::setbuf is that setbuf(0, 0) sets the stream to unbuffered output, and even then I wouldn't count on it.

The actual behavior of setbuf varies wildly from implementation to implementation:

  • libstdc++: setbuf only works if called before the file is opened. Beyond that, it does what you would probably expect. Every time the buffer fills you'll get one call to the underlying write syscall.
  • libc++: setbuf can be called after opening a file, but before any I/O is done. Every time the buffer fills you get a call to fwrite on an underlying FILE*. That means that the output is still buffered using the FILE's internal buffer. There's no way to access that internal FILE* to setbuf or setvbuf on it, so you're stuck with the default buffer size (currently 4096 bytes, in glibc's implementation at least).
  • MSVCRT: basic_filebuf shares its buffers with its underlying FILE object. setbuf just passes the buffer you give it on to a call to setvbuf on an underlying FILE*. May be called at any time, but will discard any previously-buffered data.

Upvotes: 0

catnip
catnip

Reputation: 25408

In WriteCPP, you have to set the buffer before opening the file, like so:

std::ofstream file;
char buffer[BUFF]; file.rdbuf()->pubsetbuf(buffer, BUFF);   // set buffer
file.open ("test.txt", std::ios::binary);                   // open file

Then you get the sort of results that you might you expect (times are for writing 20MB with the buffer sizes shown):

writeCPP, 32: 2.15278
writeCPP, 128: 1.21372
writeCPP, 512: 0.857389

I also benchmarked WriteC with your change from fprintf to fputc and got the following (again writing 20MB):

writeC, 32: 1.41433
writeC, 128: 0.524264
writeC, 512: 0.355097

Test program is here:

https://wandbox.org/permlink/F2H2jcrMVsc5VNFf

Upvotes: 3

jamesdlin
jamesdlin

Reputation: 90175

fprintf has extra overhead since it needs to scan its input string for format specifiers, so you're not quite doing an apples-to-apples comparison.

A better comparison would be to use fputs instead of fprintf or to use fputc and then use file << 'a' in the iostream version.

Upvotes: 2

Related Questions