QDataStream and Flush

Question

This is a QT question about using QDataStream and QTemporaryFile in C++ and Linux.

I am having some issues getting a QDataStream to flush. QTextStream has a flush function, however QDataStream apparently doesn't need one. (Citation from 2013: http://www.qtcentre.org/threads/53042-QDataStream-and-flush()). My question is, is this actually/still the case, and is there anyway to force a QDataStream to flush?

When I process the file that I have written using QDataStream the last number of writes are missing (112 Bytes when writing 5 bytes at a time, 22 bytes when writing 1 Byte at a time). However, if I write a large quantity of padding to the end of the file then all of the contents is present (except for the last few writes of the padding). This is why I believe the QDataStream is not being flushed to the file.

The files I am processing are medium size raw binary files (Around 2MB).

Here a minimal example using some of the code I have for processing the files:

void read_and_process_file(QString &filename) {
  QFile inputFile(filename);
  if (!inputFile.open(QIODevice::ReadOnly)) {
    qDebug() << "Couldn't open: " << filename;
    return;
  }
  QDataStream fstream(&inputFile);
  QTemporaryFile *tempfile = new QTemporaryFile();
  if (!tempfile->open()) {
    qDebug() << "Couldn't open tempfile";
    return;
  }
  QDataStream ostream(tempfile);

  while (!fstream.atEnd()) {
    int block_size = 5;      //The number to read at a time
    char lines[block_size];

    //Read from the input file
    int len = fstream.readRawData(lines,block_size);
    QByteArray data(lines,len);

    //Will process data here once copying works

    //Write to the temporary file
    ostream.writeRawData(data,data.size());
  }
  process_file(tempfile);
  delete tempfile;
}

Answer 1

The first part of this answer is independent of the issue of flushing the file to disk.

---

Use of !fstream.atEnd() as the conditional of while is not a good idea. See http://stackoverflow.com/questions/5605125/why-is-iostreameof-inside-a-loop-condition-considered-wrong. I would change the while loop to:

const int block_size = 5;      //The number to read at a time
char lines[block_size];
int len = 0;
while ( (len = fstream.readRawData(lines,block_size)) > 0) {

    QByteArray data(lines, len);

    //Will process data here once copying works

    //Write to the temporary file
    ostream.writeRawData(data,data.size());
}

However, I don't see the point of using the intermediate QByteArray. That loop can be simplified to:

while ( (len = fstream.readRawData(lines,block_size)) > 0) {
    //Write to the temporary file
    ostream.writeRawData(lines, len);
}

If you need to process the QByteArray for other things, it's fine to construct one and use it but the call to ostream.writeRawData doesn't need to use it.

---

Re. the issue of the file not getting flushed, I would suggest using a nested scope to open the file. The file should get flushed and closed at the end of the scope.

void read_and_process_file(QString &filename) {

   QFile inputFile(filename);
   if (!inputFile.open(QIODevice::ReadOnly)) {
      qDebug() << "Couldn't open: " << filename;
      return;
   }

   QDataStream fstream(&inputFile);
   QTemporaryFile *tempfile = new QTemporaryFile();
   if (!tempfile->open()) {
      qDebug() << "Couldn't open tempfile";
      return;
   }

   // Create a nested scope for the QDataStream
   // object so it gets flushed and closed when the 
   // scope ends.
   {
      QDataStream ostream(tempfile);

      const int block_size = 5;      //The number to read at a time
      char lines[block_size];
      int len = 0;
      while ( (len = fstream.readRawData(lines,block_size)) > 0) {

         QByteArray data(lines, len);

         //Will process data here once copying works

         //Write to the temporary file
         ostream.writeRawData(lines, len);
      }

      // The QDataStream should be flushed and
      // closed at the end of this scope.
   }

   process_file(tempfile);
   delete tempfile;
}