Reputation: 569
I need to parse this html code using HtmlAgilityPack and C#. I can get the div class="patent_bibdata" node, but I don'know how to loop thru the child nodes.
In this sample there are 6 hrefs, but I need to separate them into two groups; Inventors, Classification. I'm not interested in the last two. There can be any number of hrefs in this div.
As you can see there is a text before the two groups that says what the hrefs are.
code snippet
HtmlWeb hw = new HtmlWeb();
HtmlDocument doc = m_hw.Load("http://www.google.com/patents/US3748943");
string xpath = "/html/body/table[@id='viewport_table']/tr/td[@id='viewport_td']/div[@class='vertical_module_list_row'][1]/div[@id='overview']/div[@id='overview_v']/table[@id='summarytable']/tr/td/div[@class='patent_bibdata']";
HtmlNode node = m_doc.DocumentNode.SelectSingleNode(xpath);
So how would you do this?
<div class="patent_bibdata">
<b>Inventors</b>:
<a href="http://www.google.com/search?tbo=p&tbm=pts&hl=en&q=ininventor:%22Ronald+T.+Lashley%22">
Ronald T. Lashley
</a>,
<a href="http://www.google.com/search?tbo=p&tbm=pts&hl=en&q=ininventor:%22Ronald+T.+Lashley%22">
Ronald T. Lashley
</a><br>
<b>Current U.S. Classification</b>:
<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200P">84/312.00P</a>;
<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200R">84/312.00R</a><br>
<br>
<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://patft.uspto.gov/netacgi/nph-Parser%3FSect2%3DPTO1%26Sect2%3DHITOFF%26p%3D1%26u%3D/netahtml/PTO/search-bool.html%26r%3D1%26f%3DG%26l%3D50%26d%3DPALL%26RefSrch%3Dyes%26Query%3DPN/3748943&usg=AFQjCNGKUic_9BaMHWdCZtCghtG5SYog-A">
View patent at USPTO</a><br>
<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://assignments.uspto.gov/assignments/q%3Fdb%3Dpat%26pat%3D3748943&usg=AFQjCNGbD7fvsJjOib3GgdU1gCXKiVjQsw">
Search USPTO Assignment Database
</a><br>
</div>
Wanted result InventorGroup =
<a href="http://www.google.com/search?tbo=p&tbm=pts&hl=en&q=ininventor:%22Ronald+T.+Lashley%22">
Ronald T. Lashley
</a>
<a href="http://www.google.com/search?tbo=p&tbm=pts&hl=en&q=ininventor:%22Ronald+T.+Lashley%22">
Thomas R. Lashley
</a>
ClassificationGroup
<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200P">84/312.00P</a>;
<a href="http://www.google.com/url?id=3eF8AAAAEBAJ&q=http://www.uspto.gov/web/patents/classification/uspc084/defs084.htm&usg=AFQjCNEZRFtAyKTfNudgc-XVt2-VboD77Q#C084S31200R">84/312.00R</a>
The page I'm trying to scrape: http://www.google.com/patents/US3748943
// Anders
PS! I know that in this page the names of the inventors are the same, but in most of them they are different!
Upvotes: 3
Views: 3212
Reputation: 569
So it's obvious that I don't understand XPath (yet). So I came up with this solution. Maybe not the smartest solution, but it works!
// Anders
List<string> inventorList = new List<string>();
List<string> classificationList = new List<string>();
string xpath = "/html/body/table[@id='viewport_table']/tr/td[@id='viewport_td']/div[@class='vertical_module_list_row'][1]/div[@id='overview']/div[@id='overview_v']/table[@id='summarytable']/tr/td/div[@class='patent_bibdata']";
HtmlNode nodes = m_doc.DocumentNode.SelectSingleNode(xpath);
bool bInventors = false;
bool bClassification = false;
for (int i = 0; i < nodes.ChildNodes.Count; i++)
{
HtmlNode node = nodes.ChildNodes[i];
string txt = node.InnerText;
if (txt.IndexOf("Inventor") > -1)
{
bClassification = false;
bInventors = true;
}
if (txt.IndexOf("Classification") > -1)
{
bClassification = true;
bInventors = false;
}
if (txt.IndexOf("USPTO") > -1)
{
bClassification = false;
bInventors = false;
}
string name = node.Name;
if (name.IndexOf("a") > -1)
{
if (bInventors)
{
string inventor = node.InnerText;
inventorList.Add(inventor);
}
if (bClassification)
{
string classification = node.InnerText;
classificationList.Add(classification);
}
}
Upvotes: 2
Reputation: 139306
XPATH is your friend! Something like this will get you the inventors name:
HtmlWeb w = new HtmlWeb();
HtmlDocument doc = w.Load("http://www.google.com/patents/US3748943");
foreach (HtmlNode node in doc.DocumentNode.SelectNodes("//div[@class='patent_bibdata']/br[1]/preceding-sibling::a"))
{
Console.WriteLine(node.InnerHtml);
}
Upvotes: 5