CODEWITHSUNDEEP
c++qtgumbo
Dmitry K.
Dmitry K.

Reputation: 353

Gumbo HTML text inside A

I'm using Gumbo to parse a web page in CP1251. I've converted the text into UTF-8 and sent it to gumbo parser. I have problem with getting text inside A link with

node->v.text.text

I get strange symbols on the output while the source is correctly displayed in the console. I am using Qt 5.2 and libiconv for converting purposes.

Need I convert node text to local code page or what am I doing wrong?

Getting page in CP1251

    QByteArray barrData = pf->getData();

    size_t dstlen = 1048576;
    char buf[dstlen];
    memset((char*)buf, 0, dstlen);

    char* pIn = barrData.data();
    char* pOut = (char*)buf;

    size_t srclen = barrData.size();


    iconv_t conv = iconv_open("UTF-8", "CP1251");
    iconv(conv, &pIn, &srclen, &pOut, &dstlen);
    iconv_close(conv);

    GumboOutput* output = gumbo_parse(buf);

    parsePage(output->root);
    gumbo_destroy_output(&kGumboDefaultOptions, output);

Parsing

if (node->v.element.tag == GUMBO_TAG_DIV && (_class = gumbo_get_attribute(&node->v.element.attributes, "class")))
{
    if (QString(_class->value) == "catalog-item-title")
    {
        qDebug() << "parsePage: found product, parsing...";

        GumboVector* children = &node->v.element.children;
        for (int i = 0; i < children->length; ++i)
        {
            GumboNode* node = static_cast<GumboNode*>(children->data[i]);

            GumboAttribute* href;
            GumboAttribute* id;

            if (node->v.element.tag == GUMBO_TAG_A &&
                (href = gumbo_get_attribute(&node->v.element.attributes, "href"))
            )
            {
                char buf[1024];
                memset(buf, 0, 1024);
                int i = node->v.text.original_text.length;
                memcpy(buf, node->v.text.original_text.data, i);


                QString strTitle = buf;
                Q_ASSERT(node->v.text.original_text.length > 0);
                qDebug() << "parsePage: found product" << strTitle << href->value;

                break;
            }
        }
    }
}

Source page text:

<div class="catalog-item-title"><a href="/textile/postelnoe-bele/korolevskoe-iskushenie-perkal/izmir_2/">Измир 2</a></div>

Upvotes: 3

Views: 3319

Answers (1)

Dmitry K.
Dmitry K.

Reputation: 353

I have smoked up examples at last. The text is contained inside child node.

if (node->v.element.tag == GUMBO_TAG_A &&
    (href = gumbo_get_attribute(&node->v.element.attributes, "href"))
)
    {
    QString strTitle;
    GumboNode* title_text = static_cast<GumboNode*>(node->v.element.children.data[0]);
    if (title_text->type == GUMBO_NODE_TEXT)
    {
        strTitle = title_text->v.text.text;
    }

    qDebug() << "parsePage: found product" << strTitle << href->value;

    break;
}

Upvotes: 6

Related Questions