CODEWITHSUNDEEP
vb.nethtml-agility-pack
gromit1
gromit1

Reputation: 587

How to extract data from a HTML table?

I recently downloaded HtmlAgilityPack but I haven't found any real instructions on how to use it. I have attempted to piece together some code based on some various discussion board posts and other sources. Here is what I have so far:

Private Sub Button3_Click(ByVal sender As System.Object, ByVal e As System.EventArgs)
    Dim document As New HtmlAgilityPack.HtmlDocument
    document.LoadHtml("www.reuters.com/finance/stocks/overview?symbol=GOOG")

    Dim tabletag = document.DocumentNode.SelectSingleNode("//table[@class='data']/tr[1]/td[2]")
End Sub

As you can see I am working with the HTML from www.reuters.com/finance/stocks/overview?symbol=GOOG.

I am trying to extract the Beta value from this page. This value is currently 1.04.

When I run the code above my immediate window shows this repeated 100 times:

1.04
$243,156.41
328.59
--
--
Trading Report for (GOOG). A detailed report, including free correlated market analysis, and updates.
ValuEngine Detailed Valuation Report for GOOG
GOOGLE INC CL A (GOOG)  12-months forecast
GOOGLE INC CL A (GOOG)  2-weeks forecast
Google Inc: Business description, financial summary, 3yr and interim financials, key statistics/ratios and historical ratio analysis.

I only want the first number (1.04) returned. What am I doing wrong? Any suggestions?

Upvotes: 3

Views: 6621

Answers (1)

chrixbittinx
chrixbittinx

Reputation: 71

You need to use cookies and a proxy. The below works great for me. Let me know your thoughts:

Imports System.Net
Imports System.Web

Public Class Form1

    Public cookies As New CookieContainer

    Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click


        Dim wreq As HttpWebRequest = WebRequest.Create("http://www.reuters.com/finance/stocks/overview?symbol=GOOG")

        wreq.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.5) Gecko/20091102 Firefox/3.5.5"

        wreq.Method = "get"

        Dim prox As IWebProxy = wreq.Proxy

        prox.Credentials = CredentialCache.DefaultCredentials


        Dim document As New HtmlAgilityPack.HtmlDocument
        Dim web As New HtmlAgilityPack.HtmlWeb

        web.UseCookies = True
        web.PreRequest = New HtmlAgilityPack.HtmlWeb.PreRequestHandler(AddressOf onPreReq)

        wreq.CookieContainer = cookies

        Dim res As HttpWebResponse = wreq.GetResponse()


        document.Load(res.GetResponseStream, True)

        'just for testing:
        '   Dim tabletag = document.DocumentNode.SelectNodes("//table")
        '  MsgBox(tabletag.Nodes.Count.ToString)

        'returns your field
        Dim tabletag2 = document.DocumentNode.SelectSingleNode("//td[@class='data']")
        MsgBox(tabletag2.InnerText)

    End Sub

    Private Function onPreReq(req As HttpWebRequest)

        req.CookieContainer = cookies
        Return True

    End Function
End Class

Upvotes: 2

Related Questions