CODEWITHSUNDEEP
pythonxpathhtml-tableextract
Philip McQuitty
Philip McQuitty

Reputation: 1077

Extract html cell data XPath

I have this simple html table: 

<tr align="center" class="tableRow1Font" >
<td>OPEN</td>
<td>80002</td>
<td>
<span style="font-weight:bold;">
ACCY
</span> 
<A HREF="http://bulletin.gwu.edu/search/?P=ACCY+2001" target="_blank">
<span style="font-weight:bold;">
2001
</span>
</A>
</td>

<td>10</td>
<td>Intro Financial Accounting</td>
<td>3.00</td>
<td> Ray, K</td>
<td><a href="http://virtualtour.gwu.edu/#MON" target="_blank" >MON</a> 113</td>
<td>MW<br>12:45PM - 02:00PM</td>
<td>08/25/14 - 12/06/14</td>
<td>

</td>
</tr>

I am using xpath and requests with Python. Using the xpath function how would I extract all the 'td' titles out of this table and separate them with a comma?

I want the extracted data to look like this:

OPEN, 80002, ACCY 2001, 10, Intro to Financial Accounting, 3.00, Ray, K, MW 12:45-02:00PM

Upvotes: 0

Views: 76

Answers (1)

WGS
WGS

Reputation: 14169

Try this:

Code:

src = """<tr align="center" class="tableRow1Font" >
<td>OPEN</td>
<td>80002</td>
<td>
<span style="font-weight:bold;">
ACCY
</span> 
<A HREF="http://bulletin.gwu.edu/search/?P=ACCY+2001" target="_blank">
<span style="font-weight:bold;">
2001
</span>
</A>
</td>
<td>10</td>
<td>Intro Financial Accounting</td>
<td>3.00</td>
<td> Ray, K</td>
<td><a href="http://virtualtour.gwu.edu/#MON" target="_blank" >MON</a> 113</td>
<td>MW<br>12:45PM - 02:00PM</td>
<td>08/25/14 - 12/06/14</td>
<td>
</td>
</tr>"""

from lxml import html

tree = html.fromstring(src)
tds = tree.xpath("//td/descendant-or-self::*/text()[normalize-space()]")

print ", ".join([td.strip() for td in tds])

Result:

OPEN, 80002, ACCY, 2001, 10, Intro Financial Accounting, 3.00, Ray, K, MON, 113, MW, 12:45PM - 02:00PM, 08/25/14 - 12/06/14
[Finished in 0.5s]

Note that this gets all the text from inside all td tags, including the ones from inside the <a> child node, ie. MON.

Cleaning the result is up to you.

Upvotes: 1

Related Questions