FreddyKay
FreddyKay

Reputation: 285

Wriring binary data from QSharedPointer<float> gives unexpected file size

My issue at the moment is essentially that I can't seem to be able to write all the data that I want to disk.

What I am trying to do is saving float values in binary format to disk, one after the other. There can be any even number of values between 1.000.000 and 400.000.000 float values. Internally the data is stored in QSharedPointer. I am using the same data in different functions, in different threads. I am making sure, that the access is not changing the values in the field.

What I have is similar to this:

class blubb : public QObject
{
    Q_OBJECT
 public slots:
    void foo(QSharedPointer<float> data, size_t size)
    {
        QFile saveFile("ExampleFileName");
        QDataStream streamer;

        saveFile.open(QIODevice::WriteOnly);

        streamer.setDevice(&saveFile);
        streamer.setFloatingPointPrecision(QDataStream::SinglePrecision);
        streamer.setByteOrder(QDataStream::BigEndian);

        qDebug() << "Now writing: " << size << " elements." << endl;
        qDebug() << "With: " << sizeof (*data.data()) << " byte per entry." << endl;
        qDebug() << "Total space: " << size * sizeof(*data.data()) << " bytes.";        


        for (size_t i = 0; i < size; i++)
        {
            streamer << (data.data())[i];
        }

        qInfo() << "Writing done of " << size << " float values of "
                << sizeof (*data.data()) << " bytes per entry, total written space: "
                << size * sizeof(*data.data());
    }
};

If I have for example 4.000.000 float value in data I get all things as expected: 4.000.000 elements to write, 4 bytes per element and a total of 16.000.000 bytes to write. But the actual amount I find written is: 15.990.784 bytes.

If I try with a total of 2.000.000 elements in data the amount of bytes written is: 7.995.392 instead of 8.000.000. This is consistent and repeatable. The amount of bytes missing is proportional to the amount of entries I have in data.

Fun fact: I have another function in another thread that uses the same QSharedPointer to do this:

class blubber2 : public QObject
{
    Q_OBJECT
public slots:
    void fooOtherThread(QSharedPointer<float> data, size_t size)
    {
        if ((size % 2) != 0)
        {
            std::invalid_argument ex("Data is not evenly sized.");
            throw ex;
        }

        QVector<float> vec1;
        QVector<float> vec2;

        bool toggle = false;

        std::partition_copy(data.data(),
                            data.data()+size,
                            std::back_inserter(vec1),
                            std::back_inserter(vec2),
                            [&toggle](int)
        {
            return toggle = !toggle;
        });

    }
};

The amount of elements in vec1 and vec2 are as expected 2.000.000 if I have 4.000.000 entries in data.

So what am I doing wrong in the first function? Why am I getting a wrong amount of bytes written to file?

EDIT:// This should start the whole thing:

class Starter : public QObject
{
signals: 
    void startSignal(QSharedPointer<float> data, size_t size);

public slots:
    void helperStart()
    {
        size_t size = 2000000;
        QSharedPointer<float> data(new float[size]);
        emit startSignal(data, size);
    }

};
int main (int argc, char ** argv)
{
    QCoreApplication app(argc, argv);
    qRegisterMetaType<QSharedPointer<float>>("QSharedPointer<float>");
    qRegisterMetaType<size_t>("size_t");

    blubb blubbInstance;
    blubber2 blubberInstance;

    QThread blubbThread;
    QThread blubberThread;

    blubbInstance.moveToThread(&blubbThread);
    blubberInstance.moveToThread(&blubberThread);

    blubbThread.start();
    blubberThread.start();


    Starter starterInstance;
    QTimer timer;
    timer.setSingleShot(500);
    QObject::connect(&timer, &QTimer::timeout, &starterInstance, &Starter::helperStart);

    QObject::connect(&starterInstance, &Starter::startSignal, &blubbInstance, &blubb::foo);
    QObject::connect(&starterInstance, &Starter::startSignal, &blubberInstance, &blubber2::fooOtherThread);

    timer.start();
    app.exec();        
}

Upvotes: 0

Views: 226

Answers (0)

Related Questions