QNetworkAccessManager memory leak SSCCE - Solved

Question

I've written several apps interfacing with different devices, some of these apps use HTTP APIs. My apps need not only to interact with these devices when the connection is ok, but also inform the user whether the iteraction was successful or there was a timeout. That's why I can't wait for the OS to inform me and need to set timeouts manually.

I've written a minimal working example to show what I'm doing and to ask what am I doing wrong. The example relies on Linux for memory consumption monitoring - anyone compiling it for Window needs to comment out memory measurement part and use some other tools to do the measurement (I expect mprof to work under Windows also, but I didn't test it).

So here's the code:

#include <QByteArray>
#include <QCoreApplication>
#include <QDateTime>
#include <QDebug>
#include <QFile>
#include <QNetworkAccessManager>
#include <QNetworkReply>
#include <QScopeGuard>
#include <QTextStream>
#include <QTimer>
#include <QUrl>

QMap<QString, QString> status() {
  QFile file("/proc/self/status");
  file.open(QIODevice::ReadOnly);
  auto ba = file.readAll();
  auto list = QString::fromUtf8(ba).split("\n");
  QMap<QString, QString> retval;
  for (auto const &line : list) {
    auto keyValue = line.split(":\t");
    retval[keyValue.first()] = keyValue.last().simplified();
  }
  return retval;
}

void reportMemory() {
  auto timestamp = QDateTime::currentDateTime().toString("hh:mm:ss.zzz");
  auto memoryUsage = status()["RssAnon"];
  qDebug() << "*****" << timestamp << memoryUsage << "*****";
  QFile file("memory_log.csv");
  file.open(QIODevice::WriteOnly | QIODevice::Append);
  QTextStream stream(&file);
  stream << timestamp << ";" << memoryUsage << "\n";
}

QNetworkAccessManager qnam;
QSet<QNetworkReply *> activeReplies;

void onRequestFinished(QNetworkReply *reply) {
  if (!activeReplies.contains(reply))
    return;
  activeReplies.remove(reply);
  QScopeGuard onScopeExit([reply]() { reply->deleteLater(); });
  auto data = reply->readAll();
  qDebug() << "Reply size is" << data.size();
}

void onTimeout(QNetworkReply *reply) {
  if (!activeReplies.contains(reply))
    return;
  activeReplies.remove(reply);
  QScopeGuard onScopeExit([reply]() { reply->deleteLater(); });
  qDebug() << "Reply timeout";
}

void onRequestTimer() {
  QUrl url("http://172.20.0.10:12229");
  auto reply = qnam.get({QNetworkRequest(url)});
  activeReplies.insert(reply);
  QTimer::singleShot(500, [reply]() { onTimeout(reply); });
}

int main(int argc, char *argv[]) {
  QCoreApplication a(argc, argv);

  // Memory reporting
  QTimer memoryReportTimer;
  QObject::connect(&memoryReportTimer, &QTimer::timeout, &a, &reportMemory);
  memoryReportTimer.start(1000 * 60);
  reportMemory();

  // Network request processing
  QObject::connect(&qnam, &QNetworkAccessManager::finished, &a,
                   &onRequestFinished);
  QTimer requestTimer;
  QObject::connect(&requestTimer, &QTimer::timeout, &a, &onRequestTimer);
  requestTimer.start(1000);
  return a.exec();
}

The memory leak is around 300 kb per hour. I'm sending one request every second and writing a record in a memory log every minute. I'm keeping the list of active replies so that a success or a failure (whichever arrives first) removes the reply from the list and deletes it, so that I can be sure that each reply was deleted exactly once. I'm using one of my services as a server to connect to, so that I can connect/disconnect VPN connection to this service and see whether connection/disconnection is detected correctly. But any other service responding to GET requests will do for testing.

Valgrind wasn't a great help here. On some settings it yields nothing, on some other it produces too many false positives. So can someone please tell me what am I doing wrong or how to search for such leaks?

Solution: in Qt 5.15.2 this really is a bug, possibly related to https://bugreports.qt.io/browse/QTBUG-88063 . It's probably not the real memory leak, it's the list of or orphaned connections from QNetworkAccessManager to QNetworkReply objects that would be deleted in QNetworkAccessManager's destructor, but the QNAM is never destroyed so that list is never cleared. In newer versions of Qt this bug is supposed to be fixed - I've tried 6.8.2 and the same code works flawlessly.

Answer 1

It's a bit difficult to see what you are getting without access to whatever the server "http://172.20.0.10:12229" does.

If I make the folowing changes:

#include <QUrl>
#include <valgrind/memcheck.h>

...

void reportMemory() {
  auto timestamp = QDateTime::currentDateTime().toString("hh:mm:ss.zzz");
  auto memoryUsage = status()["RssAnon"];
  qDebug() << "*****" << timestamp << memoryUsage << "*****";
  QFile file("memory_log.csv");
  file.open(QIODevice::WriteOnly | QIODevice::Append);
  QTextStream stream(&file);
  stream << timestamp << ";" << memoryUsage << "\n";
  VALGRIND_DO_ADDED_LEAK_CHECK;
}

...

  requestTimer.start(1000);
  VALGRIND_DO_LEAK_CHECK;
  return a.exec();

I get a few leaks from glib (normal, it leaks like a sieve) and from thread creation/TLS. Fairly quickly it seems to stabilize at around 600 bytes possibly lost and 300k reachable.