Reputation: 5922
First off, I hope this question isn't too general in its scope – if so I apologize.
I'm building a web scraper using Selenium
for Python 2.7
. Previously, I have used "static" XPaths
to direct it toward certain elements. I want to implement a solution that can finds elements contextually (relative to other elements).
Let's say we wish to grab the text from the sibling element following the "Issuer:" label on this page: http://etfdb.com/etf/ROBO/. In this case the adjacent text is "Exchange Traded Concepts".
From what I've gathered, it could be possible to use a number of techniques including relative XPath
, CSS
or DOM
(?).
What would be an advisable way to go about this? Please demonstrate with code if possible.
Current "Static" XPath, where have identified the XPath with FirePath for Firefox:
try:
xpath_issuer = ".//*[@id='overview']/div/div[2]/div/div[1]/ul[1]/li[1]/span[2]/a"
find_issuer = driver.find_element_by_xpath(xpath_issuer)
issuer = re.search(r"(.+)", find_issuer.text).group().encode("utf-8")
print "Issuer: %s" % issuer
break
except NoSuchElementException:
pass
Upvotes: 1
Views: 1409
Reputation: 474281
I would make a reusable function to get the value by key:
Example working implementation:
from selenium.common.exceptions import NoSuchElementException
def get_value(driver, key):
key = key + ":"
try:
return driver.find_element_by_xpath("//span[@class='minimal-list__title' and . = '%s']/following-sibling::span" % key).text
except NoSuchElementException:
print "Not Found"
return None
Usage:
get_value(driver, "Issuer")
Upvotes: 1
Reputation: 5818
You can go with the following xpath
//span[@class='minimal-list__title'][text()='Issuer:']//following-sibling::span[@class='minimal-list__value']
Upvotes: 2