Proper way to Copy files usign QT with GUI avoiding freeze

Question

I have an application that copy files from one location to another, using QFile::Copy(..). The copy process is executed in a separated thread into a worker object, however, sometimes the GUI freezes, I have read a lot of topics here about this, but I understand that this method (worker class) is the correct one. I have the same approach on other projects to execute process in another thread like a chronometer, and it works really nice, smooth. It seems that this little freeze only happens when copying files. In MS Windows, the "lag" is more noticeable than in Linux, in the last one, no freezes are detected unless you copy a big file (700MB), but only when the file is ending the copy, during the copy, the GUI is responsive. I'm using this approach in my mainWindow (BUMain) class:

void BUMain::initThreadSetup()
{
    thread = new QThread;
    Worker *worker = new Worker();
    worker->moveToThread(thread);

    connect(worker,SIGNAL(worker_Signal_updateProgressBar(int)),ui->progressBar,SLOT(setValue(int)),Qt::QueuedConnection);
    connect(this,SIGNAL(main_signal_copyFile(int,QStringList,QString)),worker,SLOT(worker_Slot_copyFile(int,QStringList,QString)),Qt::QueuedConnection);
    connect(worker,SIGNAL(worker_signal_keepCopying()),this,SLOT(main_slot_keepCopying()),Qt::QueuedConnection);
    connect(worker,SIGNAL(worker_signal_logInfo(QString)),gobLogViewer,SLOT(logger_slot_logInfo(QString)),Qt::QueuedConnection);

    connect(thread,SIGNAL(finished()),worker,SLOT(deleteLater()));
    connect(thread,SIGNAL(finished()),thread,SLOT(deleteLater()));
    thread->start();

}

This method is called in the MainGui (BUMain) constructor to start the thread. Some relevant code:

This is the slot called when worker_signal_keepCopying() is emited:

void BUMain::main_slot_keepCopying()
{
    giProgress ++;
    if(giProgress < giFileCounter){
        emit(main_signal_copyFile(giProgress,gobPaths,ui->toFilesTextField->text()));
    }
}

Here, I do a counter validation and emit a new signal to inform the worker that it can continue with the next copy. The copy process is done one by one file. In the worker.cpp file you can see the slot worker_Slot_copyFile(int liIndex,QStringList files,QString path) implementation:

The worker.cpp:

#include "worker.h"
#include <QFile>
#include <QFileInfo>
#include <QStringList>
#include <QCoreApplication>

Worker::Worker(QObject *parent) :
    QObject(parent)
{

}

void Worker::worker_Slot_copyFile(int liIndex,QStringList files,QString path)
{
    QString fileName;

    fileName = QFileInfo(files.at(liIndex)).baseName()+"."+QFileInfo(files.at(liIndex)).completeSuffix();

    //If the file exist, delete it
    if (QFile::exists(path+"/"+fileName))
    {
        QFile::remove(path+"/"+fileName);
    }

    QFile lobFile(files.at(liIndex));

    if(lobFile.copy(path+"/"+fileName)){

        //Write to a logger class
        emit(worker_signal_logInfo("File: " + fileName + " copied to: " + path));

        //Update a progress bar in the main GUI
        emit(worker_Signal_updateProgressBar(liIndex+1));
    }

    //The file has been processed!, I'm ready to copy another file...
    emit(worker_signal_keepCopying());

}

worker.h:

#ifndef WORKER_H
#define WORKER_H

#include <QObject>
#include <QThread>
#include <QStringList>

class Worker : public QObject
{
    Q_OBJECT
public:
    explicit Worker(QObject *parent = 0);

signals:
    void worker_Signal_updateProgressBar(int value);
    void worker_signal_keepCopying();
    void worker_signal_logInfo(QString info);

public slots:
    void worker_Slot_copyFile(int liIndex, QStringList files, QString path);

};

#endif // WORKER_H

So, in words the process could be:

Ok! lets start copying some files!!. Remember BUMain is the mainWindow class where the GUI is running:

1. BUMain adjust the progress bar maximum to the total of files to copy
2. BUMain set the progress bar value to 0
3. BUMain emit the main_signal_copyFile(...) signal. This is like "Hey worker! start copy a new file please, I will be here doing another stuff, and when you finish copying it, plase tell me."
4. Worker receives the signal and invokes worker_Slot_copyFile. Is like "Hey BUMain, I can hear you, order received! Now I will copy the file"
5. Worker finishes the copy emiting worker_signal_keepCopying(): "BUMain I have ended copying the file, I can copy another if you want, just notify me."
6. BUMain invokes main_slot_keepCopying(): "I can hear you worker, Thanks!" emits main_signal_copyFile(...) again: "Worker, I have more files to copy, please copy another one".
7. The process repeats until all the files has been processed.

Sometimes, the process works really nice, no lag or freeze, but sometimes does not. Please, note that this approach is intended to be NOT BLOCKING.

I have also tried run a for loop inside the worker class to copy all the files without notify the main class, but the lag is heavy and the GUI becomes unresponsive when copying big files ( > 300MB). For example:

void Worker::worker_Slot_copyFile(int liIndex,QStringList files,QString path)
{
    QString fileName;
    for(int liIndex = 0; liIndex < files.length() - 1; liIndex ++){
        fileName = QFileInfo(files.at(liIndex)).baseName()+"."+QFileInfo(files.at(liIndex)).completeSuffix();

        if (QFile::exists(path+"/"+fileName))
        {
            QFile::remove(path+"/"+fileName);
        }

        QFile lobFile(files.at(liIndex));

        if(lobFile.copy(path+"/"+fileName)){
            emit(worker_signal_logInfo("File: " + fileName + " copied to: " + path));
            emit(worker_Signal_updateProgressBar(liIndex+1));
        }
    }
}

I hope to be as clear as possible, it is a bit difficult to explain this. I have taken this as reference to work with the worker class approach.

NOTE: I'm using QT5.1.1 to program and Windows 10, and Arch Linux to deploy the app.

Any help or advise is appreciated. Thanks in advance, and have a nice day!

Answer 1

Your approach is correct, if a bit verbose. The "lag" is likely due to the kernel being silly about how it manages the page cache and evicting parts of your application to make room for pages from the files being copied. The only remedy for that might be to use a platform-specific file copying mechanism that advises the kernel that you'll be reading and writing sequentially, and that you don't need the data anymore after you've done the writing. See this answer for details and some Linux code. On Windows, one would hope that CopyFileEx does the correct advisory calls, a Qt example is here.