Qt - How to create a QFuture from a thread

Question

I am currently working on an editor program; there's a feature I need to write, which requires loading several files in a row using the project's asynchronous file API, then performing some more computations once those files are loaded.

In another language, this would probably be implemented with an async/await workflow, eg:

let firstFile = await FileAPI.loadFile("Foo.xxx", ...);
let otherFile = await FileAPI.loadFile("Bar/Foobar.xxx", ...);

The Qt equivalent to this code would be to spawn a new thread using QtConcurrent::run, returning a QFuture, and waiting for that future to yield a result.

However, in the project I work on, the file-opening API runs on a single worker thread, which means I can't use QtConcurrent::run. This is an established, non-negotiable part of the codebase. Eg the constructor of the file API looks like:

FileApiWorker* worker = new FileApiWorker();
m_workerThread = new QThread();
worker->moveToThread( m_workerThread );

// Input signals
connect( this, &FileApi::loadFile, worker, &FileApiWorker::loadFile);
connect( this, &FileApi::loadData, worker, &FileApiWorker::loadData);
connect( this, &FileApi::loadDir, worker, &FileApiWorker::loadDir);

Which means my only way of accessing filesystem data is to call a method which emits a signal, which starts the computation on another thread, which eventually emits its own signal at the end to pass on the loaded data.

This is extremely impractical for the use case above, because instead of saying "do thing, load data, wait, keep doing things", I essentially need to say "do thing, load data (with call back 'keep doing things')" and "keep doing things" in another function, which introduces all sorts of brittleness in the code. (and, well, you know, that's exactly the sort of workflow we invented futures for)

Is there some way I could create a QFuture, or some future-equivalent object (that can be awaited inside a method) from the loadFile method, given that loadFile always runs on the same worker thread and I am not allowed to create new threads?

Answer 1

The simplest way to create a QFuture in Qt is with the undocumented QFutureInterface class.

Example code:

Q_DECLARE_METATYPE( QFutureInterface<FileData> );

// ...

qRegisterMetaType<QFutureInterface<FileData>>();

FileApiWorker* worker = new FileApiWorker();
connect( this, &FileApi::loadFile_signal, worker, &FileApiWorker::loadFile_signal);

// ...

QFuture<FileData> FileApi::loadFile()
{
    QFutureInterface<FileData> futureInterface;
    // IMPORTANT: This line is necessary to be able to wait for the future.
    futureInterface.reportStarted();

    emit loadFile_signal(futureInterface);
    return futureInterface.future();
}

FileApiWorker::loadFile_signal(QFutureInterface<FileData>& futureInterface)
{
    // Do some things
    // ...
    futureInterface.reportResult(...);
    // IMPORTANT: Without this line, future.waitForFinished() never returns.
    futureInterface.reportFinished();
}

Some factors to account for:

• The above code uses Q_DECLARE_METATYPE; which is necessary to be able to pass QFutureInterface through a cross-threads signal. To be precise, the connect line will fail to compile if Q_DECLARE_METATYPE isn't included; and the emit loadFile_signal line will fail at runtime if qRegisterMetaType isn't called. See the Qt documentation on metatypes for details.

• You can propagate errors, in such a way that calling loadFile().waitForFinished() throws on error. To achieve this, you need to create a special-purpose class inheriting QException, then call:

  futureInterface.reportException( MyException(...) );
  futureInterface.reportFinished();
  

  in your error path.

  QException is essentially a wrapper for actual exceptions that need to be transferred between threads. See the documentation for details.

• While QFutureInterface is stable, and mostly has the same API as QFuture and QFutureWatcher, it's still an undocumented feature, which may surprise contributors coming across it in a shared codebase. The class can be counter-intuitive, and fail silently if you don't respect the points above (which I had to learn through trial and error). This must be stressed in the comments of any shared code using QFutureInterface. The class's source code can be found here.

Answer 2

IMO, it is strange not to use ready-to-use solutions (AsyncFuture) and try to rewrite from scratch.

But I can suggest my own "wheel": lambda as a slot.

void FileApi::awaitLoadFile()
{
    qDebug() << "\"await\" thread is" << thread(); 

    emit loadFile("Foo.xxx");

    static bool once = connect(m_worker, &FileApiWorker::loadFileDone, this, // there is possible to avoid the third "this" parameter, but it is important to specify the lifetime of the connection and the receiver context while using lambdas
        [=](QByteArray result)
    {
        qDebug() << "\"comeback-in-place\" thread is" << thread(); // will be the same as "await" thread was

        // do what you need with your result
    },
    Qt::QueuedConnection // do not forget
    );

    qDebug() << "here is an immediate return from the \"await\" slot";
}

Useful arcticle New Signal Slot Syntax - Qt Wiki