CODEWITHSUNDEEP
htmlqtqnetworkaccessmanagerqnetworkrequestqnetworkreply
Alaa Salah
Alaa Salah

Reputation: 1067

Qt - Getting source (HTML code) of a web page hosted on the internet

I want to get the source (HTML) of a webpage, for example the homepage of StackOverflow.

This is what I've coded so far:

QNetworkAccessManager manager;
QNetworkReply *response = manager.get(QNetworkRequest(QUrl(url)));

QString html = response->readAll(); // Source should be stored here

But nothing happens! When I try to get the value of the html string it's empty ("").

So, what to do? I am using Qt 5.3.1.

Upvotes: 7

Views: 13035

Answers (4)

juzzlin
juzzlin

Reputation: 47945

A short answer including the essential part in C++17:

const auto manager = new QNetworkAccessManager(this);
connect(manager, &QNetworkAccessManager::finished,
        this, [](auto reply) {
            qDebug() << reply->readAll();
        });
manager->get(QNetworkRequest({ "https://www.google.com" }));

Upvotes: 1

Kuba hasn&#39;t forgotten Monica
Kuba hasn&#39;t forgotten Monica

Reputation: 98505

You need to code it in asynchronous fashion. C++11 and Qt come to the rescue. Just remember that the body of the lambda will execute later from the event loop.

// https://github.com/KubaO/stackoverflown/tree/master/questions/html-get-24965972
#include <QtNetwork>
#include <functional>

void htmlGet(const QUrl &url, const std::function<void(const QString&)> &fun) {
   QScopedPointer<QNetworkAccessManager> manager(new QNetworkAccessManager);
   QNetworkReply *response = manager->get(QNetworkRequest(QUrl(url)));
   QObject::connect(response, &QNetworkReply::finished, [response, fun]{
      response->deleteLater();
      response->manager()->deleteLater();
      if (response->error() != QNetworkReply::NoError) return;
      auto const contentType =
            response->header(QNetworkRequest::ContentTypeHeader).toString();
      static QRegularExpression re("charset=([!-~]+)");
      auto const match = re.match(contentType);
      if (!match.hasMatch() || 0 != match.captured(1).compare("utf-8", Qt::CaseInsensitive)) {
         qWarning() << "Content charsets other than utf-8 are not implemented yet:" << contentType;
         return;
      }
      auto const html = QString::fromUtf8(response->readAll());
      fun(html); // do something with the data
   }) && manager.take();
}

int main(int argc, char *argv[])
{
   QCoreApplication app(argc, argv);
   htmlGet({"http://www.google.com"}, [](const QString &body){ qDebug() << body; qApp->quit(); });
   return app.exec();
}

Unless you're only using this code once, you should put the QNetworkManager instance as a member of your controller class, or in the main, etc.

Upvotes: 9

MKAROL
MKAROL

Reputation: 316

You have to add QEventLoop between.

QNetworkAccessManager manager;
QNetworkReply *response = manager.get(QNetworkRequest(QUrl(url)));
QEventLoop event;
connect(response,SIGNAL(finished()),&event,SLOT(quit()));
event.exec();
QString html = response->readAll(); // Source should be stored here

Upvotes: 7

Pavel Strakhov
Pavel Strakhov

Reputation: 40512

QNetworkAccessManager works asynchronously. You call readAll() immediately after get(), but the request has not been made in that moment. You need to use QNetworkAccessManager::finished signal as shown in the documentation and move readAll() to the slot connected to this signal.

Upvotes: 6

Related Questions