CODEWITHSUNDEEP
c++linuxbash
obl
obl

Reputation: 1799

C++ much faster than Bash script writing to text file

I wanted to test the performance of writing to a file in a bash script vs a C++ program.

Here is the bash script:

#!/bin/bash

while true; do
        echo "something" >> bash.txt
done

This added about 2-3 KB to the text file per second.

Here is the C++ code:

#include <iostream>
#include <fstream>

using namespace std;

int main() {
    ofstream myfile;
    myfile.open("cpp.txt");

    while (true) {
        myfile << "Writing this to a file Writing this to a file \n";
    }

    myfile.close();
}

This created a ~6 GB text file in less than 10 seconds.

What makes this C++ code so much faster, and/or this bash script so much slower?

Upvotes: 17

Views: 5434

Answers (3)

Elliott Frisch
Elliott Frisch

Reputation: 201439

As others have already pointed out, this is because you are currently opening and closing the file with each line you write in your script (and shell scripts are interpreted while C++ is compiled). You might batch the writes instead and write once, for example

MSG="something"
logfile="test.txt"
(
for i in {1..10000}; do
        echo $MSG
done
) >> $logfile

Which will write the message 10k times but only open the log once.

Upvotes: 7

iehrlich
iehrlich

Reputation: 3592

There are several reasons to it.

First off, interpreted execution environments (like bash, perl alongside with non-JITed lua and python etc.) are generally much slower than even poorly written compiled programs (C, C++, etc.).

Secondly, note how fragmented your bash code is - it just writes a line to a file, then it writes one more, and so on. Your C++ program, on the other side, performs buffered write - even without your direct efforts to it. You might see how slower will it run if you substitute

myfile << "Writing this to a file Writing this to a file \n";

with

myfile << "Writing this to a file Writing this to a file" << endl;

for more information about how streams are implemented in C++, and why \n is different from endl, see any reference documentation on C++.

Thirdly, as comments prove, your bash script performs open/close of the target file for each line. This implies a significant performance overhead in itself - imagine myfile.open and myfile.close moved inside your loop body!

Upvotes: 39

Reece Ward
Reece Ward

Reputation: 29

Compiled vs. Interpreted Languages

Bash is interpreted while C++ is compiled. Just that makes it a lot faster

Upvotes: -3

Related Questions