CODEWITHSUNDEEP
xpathweb-scrapingscrapy
Arthur Walker
Arthur Walker

Reputation: 125

Selecting with Xpath in Scrapy

I'm using Scapy to scrape some data from a site and I need help using Xpath to select "data" from the following.

<span class="result_item"><span class="text3"><span class="header_text3">**data**</span><br />
  <a href="http://website.htm">**data**</a><br />
     **data**</span> <span class="phone_button_out"><span class="phone_button" style="margin-top: 0"
    onclick="pageTracker._trackEvent('USDSearch','Call Now!F');phone_win.open('name','**data**',27101650,0)">
  Call Now!<br />
</span></span>

What statements can I use to select the necessary data? I hope this isn't a stupid question. If it is, please point me in the right direction.

Upvotes: 0

Views: 551

Answers (1)

parishodak
parishodak

Reputation: 4676

There are multiple data elements to get in the posted html. Assuming that <span class="result_item"> is parent of the items, you can try the following:

To get header:

//span[@class='result_item']/span[@class='header_text3']/text()

To get anchor link data:

//span[@class='result_item']/a/text()

Also, to help with xpaths, install Firebug Addon in Firefox, then FirePath addon on Firebug. Pointing to elements will give you autogenerated xpaths (good for beginners. sometime needs xpath tuning)

Upvotes: 1

Related Questions