CODEWITHSUNDEEP
c++qtfileqfile
kafman
kafman

Reputation: 2860

QFile: multiple handles to same physical file do not write all data

I was wondering how QFile behaves when multiple handles are opened to the same file (using C++ in Visual Studio 2013 on Windows 7), so I wrote the following little program:

  QFile file("tmp.txt");
  file.open(QIODevice::WriteOnly | QIODevice::Truncate);
  QTextStream ts(&file);
  ts << "Hallo\n";

  QFile file2("tmp.txt");
  file2.open(QIODevice::WriteOnly | QIODevice::Append);
  QTextStream ts2(&file2);
  ts2 << "Hallo 2\n";

  file.close();

  ts2 << "Hello again\n";

  file2.close();.

This produces the following output in file tmp.txt:

Hallo 2
Hello again

So the first Hallo statement got lost. If I do a ts.flush() right after the ts << "Hallo\n" this does not happen, which makes me think that the statement got lost in the internal buffers of QString or it was overwritten by the subsequent output statements. However, I want to use QFile in a logging framework, so I don't want to always flush, as this would decrease the performance.

I also tried the same thing with std::basic_ostream<char> instead of QFile:

  std::basic_ofstream<char> file;
  file.open("tmp.txt", std::ios_base::out | std::ios_base::ate | std::ios_base::app);
  file << "Hallo\n";

  std::basic_ofstream<char> file2;
  file2.open("tmp.txt", std::ios_base::out | std::ios_base::ate | std::ios_base::app);
  file2 << "Hallo 2\n";

  file.close();

  file2 << "Hello again\n";

  file2.close();

which outputs as I would expect:

Hallo
Hallo 2
Hello again

So what is the problem with the QFile example? Is QFile not intended to be used with multiple handles pointing to the same file or what is going on here exactly? I thought that my use case is quite a common one, so I'm a bit surprised to find this behaviour. I couldn't find more specifics in the Qt documentation. I've read here that Qt opens the file in shared mode, so this shouldn't be a problem.

I eventually want to use QFile for logging (where access to the function that does the actual writing is of course synchronized), but this little example worries me that some log statements might get lost on the way. Do you think it would be better to use STL streams instead of QFile?

Edit As it was pointed out, std::endl causes a flush, so I changed the STL example above to only use \n which according to here does not cause a flush. The behavior described above is unchanged, though.

Upvotes: 1

Views: 1720

Answers (3)

Ilya
Ilya

Reputation: 5557

I seems you want it both ways.

If you want several write buffers and don't want to flush them, it's hard to be sure of having all the writes in the file, and in the right order. Your small test with std::basic_ostream is not a proof: will it work with larger writes ? Will it work on other OSes ? Do you want to risk your process for a (yet unproven) speed gain ?

Upvotes: 1

AchmadJP
AchmadJP

Reputation: 903

 QFile file("tmp.txt");
  file.open(QIODevice::WriteOnly | QIODevice::Truncate);
  QTextStream ts(&file);
  ts << "Hallo\n";

  QFile file2("tmp.txt");
  file2.open(QIODevice::Append);
  QTextStream ts2(&file2);
  ts2 << "Hallo 2\n";

  file.close();

  ts2 << "Hello again\n";

  file2.close();

Try this, i changed it so the Truncate not getting invoke by WriteOnly. Silly me didn't read that one. {UPDATE}

QIODevice::WriteOnly    0x0002  The device is open for writing. Note that this mode implies Truncate.
QIODevice::Truncate 0x0008  If possible, the device is truncated before it is opened. All earlier contents of the device are lost.

Reading Source : http://doc.qt.io/qt-5/qiodevice.html#OpenModeFlag-enum

Upvotes: 0

peppe
peppe

Reputation: 22796

There are several suspicious things going on. For starters, you are introducing two levels of buffering.

  1. First and foremost QTextStream has an internal buffer, that you can flush by calling flush on it.

  2. Second, QFile is also buffering (or, better, it's using the buffered APIs from your library -- fopen, fwrite, and so on). Pass QIODevice::Unbuffered to open to make it use the unbuffered APIs (open, write, ...).

Now since this is terribly error prone, QTextStream::flush actually flushes also the underlying file device.

Also, you're passing WriteOnly | Append which doesn't make sense. It's only one of the two.

However, note that your writes may still interleave. POSIX.1-2013 says that

A write is atomic if the whole amount written in one operation is not interleaved with data from any other process. This is useful when there are multiple writers sending data to a single reader. Applications need to know how large a write request can be expected to be performed atomically. This maximum is called {PIPE_BUF}.

(On Windows I have no idea).

Upvotes: 0

Related Questions