CODEWITHSUNDEEP
c#html-agility-pack
Nathan
Nathan

Reputation: 2513

Parsing HTML using HTMLAgilityPack

I have the following HTML that I'm trying to parse using the HTML Agility Pack.

This is a snippet of the whole file that is returned by the code:

<div class="story-body fnt-13 p20-b user-gen">
    <p>text here text here text </p>
    <p>text here text here text text here text here text text here text here text text here text here text </p>
    <div  class="gallery clr bdr aln-c js-no-shadow mod  cld">
        <div>
            <ol>
                <li class="fader-item aln-c ">
                    <div class="imageWrap m10-b">
                       &#8203;<img class="http://www.domain.com/picture.png| " src="http://www.domain.com/picture.png" alt="alt text" />
                    </div>
                    <p class="caption">caption text</p>
                </li>
            </ol>
        </div>
    </div >
    <p>text here text here text text here text here text text here text here text text here text here text </p>
    <p>text here text here text text here text here text text here text here text text here text here text text here text here text </p>
    <p>text here text here text text here text here text text here text here text text here text here text text here text here text </p>
</div>

I get this snippet of code using the following (which is messy i know)

string url = "http://www.domain.com/story.html";
var webGet = new HtmlWeb();
var document = webGet.Load(url);

var links = document.DocumentNode
        .Descendants("div")
        .Where(div => div.GetAttributeValue("class", "").Contains("story-body fnt-13 p20-b user-gen")) //
        .SelectMany(div => div.Descendants("p"))
        .ToList();
int cn = links.Count;

HtmlAgilityPack.HtmlNodeCollection tl = document.DocumentNode.SelectNodes("/html[1]/body[1]/div[1]/div[2]/div[1]/div[1]/div[1]/div[2]/div[1]");
foreach (HtmlAgilityPack.HtmlNode node in tl)
{
    textBox1.AppendText(node.InnerText.Trim());
    textBox1.AppendText(System.Environment.NewLine);
}

The code loops through each p and (for now) appends it to a textbox. All is working correctly other than the div tag with the class gallery clr bdr aln-c js-no-shadow mod cld. The result of this bit of HTML is that I get the &#8203; and caption text bits.

what's the best way to omit that from the results?

Upvotes: 1

Views: 4326

Answers (2)

Simon Mourier
Simon Mourier

Reputation: 139276

XPATH is your friend. Try this and forget about that crappy xlink syntax :-)

HtmlNodeCollection tl = document.DocumentNode.SelectNodes("//p[not(@*)]");
foreach (HtmlAgilityPack.HtmlNode node in tl)
{
    Console.WriteLine(node.InnerText.Trim());
}

This expression will select all P nodes that don't have any attributes set. See here for other samples: XPath Syntax

Upvotes: 3

Jim Mischel
Jim Mischel

Reputation: 134125

It's not quite clear what you're asking. I think you're asking how to get just the direct descendants of a particular div. If that's the case, then use ChildNodes rather than Descendants. That is:

.SelectMany(div => div.ChildNodes().Where(n => n.Name == "p"))

The problem is that Descendants does a fully recursive walk of the document tree.

Upvotes: 1

Related Questions