How to link the execution of QProcess and the advancement of the QProgressBar for a very heavy computation loop

Question

I have the following bash script to be executed on a GUI via QPushButton:

 #!/bin/bash

rostopic echo -b test_LaserScan_PointCloud2_test2.bag -p /scan > test_landing_test_2.csv
rostopic echo -b test_LaserScan_PointCloud2_test2.bag -p /velodyne_points > vel_test_2.csv

The script will go through each file and extract the related .csv. The process is heavy and it takes a little bit. The problem is that there is no way to know how long it takes unless I put a QProgressBar, but I don't know how properly link the execution of the QProcess and the advancement of the QProgressBar correctly.

As of now the extraction happens successfully but the QProgressBar is not moving from 0.

Below I created a minimal verifiable example and for completeness the source code can be found here:

mainwindow.h

#include <QMainWindow>
#include <QProcess>

QT_BEGIN_NAMESPACE
namespace Ui { class MainWindow; }
QT_END_NAMESPACE

class MainWindow : public QMainWindow
{
    Q_OBJECT
    Q_PROPERTY(float progress READ progress NOTIFY progressChanged)
    Q_PROPERTY(bool running READ running NOTIFY runningChanged)
    Q_PROPERTY(bool finished READ finished NOTIFY finishedChanged)

public:
    MainWindow(QWidget *parent = nullptr);
    ~MainWindow();

    float progress();
    bool running();
    bool finished();

public Q_SLOTS:
    void startComputation();
    void finishComputation();
    void updateProgress(int value);

signals:
    void progressChanged();
    void runningChanged();
    void finishedChanged();

private slots:
    void on_executeBtn_clicked();

private:
    Ui::MainWindow *ui;
    QProcess *executeBash;

    bool m_running = false;
    int m_progressValue = 0;
    bool m_finished = false;
};

mainwindow.cpp

MainWindow::MainWindow(QWidget *parent)
    : QMainWindow(parent)
    , ui(new Ui::MainWindow)
{
    ui->setupUi(this);

this->executeBash = new QProcess(this);
    this->executeBash->setProcessChannelMode(QProcess::MergedChannels);
    /*connect(this->executeBash, &QProcess::readyReadStandardOutput, [script = this->executeBash](){
        qDebug() << "[EXEC] DATA: " << script->readAll();
    });*/
    connect(this->executeBash, QOverload<int, QProcess::ExitStatus>::of(&QProcess::finished),
          [script = this->executeBash](int exitCode, QProcess::ExitStatus exitStatus){
        qDebug() << "[EXEC] FINISHED: " << exitCode << exitStatus;
        if(script->bytesAvailable() > 0)qDebug() << "[EXEC] buffered DATA:" << script->readAll();
    });
    connect(this->executeBash, &QProcess::errorOccurred, [script = this->executeBash](QProcess::ProcessError error){
        qDebug() << "[EXEC] error on execution: " << error << script->errorString();
    });

    connect(this->executeBash, &QProcess::readyReadStandardOutput, [this, script = this->executeBash](){

         QString s = QString::fromUtf8(script->readAll());
         qDebug() << "[EXEC] DATA: " << s;
         auto match = QRegularExpression("Stage (\\d+)/(\\d+): (.*)").match(s);
         if (match.hasMatch()) {
             int x = match.captured(1).toInt();
             int y = match.captured(2).toInt();
             QString stage_info = match.captured(3);
             qDebug() << "x = " << x;
             qDebug() << "y = " << y;
             qDebug() << "info = " << stage_info;
             this->updateProgress(x * 100 / y);
         }
    });

    // Initialization of the progressbar
    m_running = false;
    emit runningChanged();
    m_finished = false;
    emit finishedChanged();
    updateProgress(0);

}

MainWindow::~MainWindow()
{
    delete ui;
}


// ProgressBag loading depending on the workload of the .sh file
float MainWindow::progress()
{
  return m_progressValue;
}

bool MainWindow::running()
{
  return m_running;
}

bool MainWindow::finished()
{
  return m_finished;
}

void MainWindow::startComputation()
{
  m_running = true;
  emit runningChanged();
  updateProgress(100);
}

void MainWindow::finishComputation()
{
  m_finished = true;
  emit finishedChanged();

  m_running = false;
  emit runningChanged();
}

void MainWindow::updateProgress(int value)
{
  m_progressValue = value;
  emit progressChanged();

  if (m_progressValue == 100)
      finishComputation();

  ui->progressBarExecuteScript->setValue(value);

}


void MainWindow::on_executeBtn_clicked()
{
    qDebug() << "Button clicked!";
    this->executeBash->start(QStringLiteral("/bin/sh"), QStringList() << QStringLiteral("/home/emanuele/catkin_docking_ws/devel/lib/test.sh")); //will start new process without blocking
    // right after the execution of the script the QProgressBar will start computing
    //ui->progressBarExecuteScript->setValue(executeBash->readAll().toInt());
}

What I have done so far

1) After setting up all the necessary points I came across this source where the user had the same problem I have now but it was resolved via QDir, I am not sure about this solution though.

2) After going through the official documentation I found out about future watcher. I am new to this tool and am having difficulties understanding how to apply it and that is the reason why I created also the minimal verifiable example.

3) I tried to setValue of the progressbar based on the execution of the bash file. Or better its advancement as shown below:

ui->progressBarExecuteScript->setValue(executeBash->readAll().toInt());

I thought that this could have solved the problem but the QProgressBar stays at 0 value and I am not sure why.

4) I consulted the QProgressBar official documentation but I didn't find anything to help me solve the problem.

Please point to the right direction to solve this issue.

Answer 1

First, modify the script to output which stage it is currently doing:

#!/bin/bash
set -e
echo "Stage 1/2: Scan"
rostopic echo -b test_LaserScan_PointCloud2_test2.bag -p /scan > test_landing_test_2.csv
echo "Stage 2/2: Velodyne"
rostopic echo -b test_LaserScan_PointCloud2_test2.bag -p /velodyne_points > vel_test_2.csv

I also added set -e to make it stop on the first failed command.

Now, you can look for these "Stage x/y" markers in the output stream and call updateProgress if so.

connect(this->executeBash, &QProcess::readyReadStandardOutput, [this, script = this->executeBash](){
     QString s = QString::fromUtf8(script->readAll());
     auto match = QRegularExpression("Stage (\\d+)/(\\d+): (.*)").match(s);
     if (match.hasMatch()) {
         int x = match.captured(1).toInt();
         int y = match.captured(2).toInt();
         QString stage_info = match.captured(3);
         this->updateProgress(x * 100 / y);
     }
});