CODEWITHSUNDEEP
pythoncssseleniumxpathselenium-webdriver
P A N
P A N

Reputation: 5922

Selenium: Find adjacent elements

First off, I hope this question isn't too general in its scope – if so I apologize.

I'm building a web scraper using Selenium for Python 2.7. Previously, I have used "static" XPaths to direct it toward certain elements. I want to implement a solution that can finds elements contextually (relative to other elements).

Let's say we wish to grab the text from the sibling element following the "Issuer:" label on this page: http://etfdb.com/etf/ROBO/. In this case the adjacent text is "Exchange Traded Concepts".

From what I've gathered, it could be possible to use a number of techniques including relative XPath, CSS or DOM(?).

What would be an advisable way to go about this? Please demonstrate with code if possible.

Current "Static" XPath, where have identified the XPath with FirePath for Firefox:

try:
    xpath_issuer = ".//*[@id='overview']/div/div[2]/div/div[1]/ul[1]/li[1]/span[2]/a"
    find_issuer = driver.find_element_by_xpath(xpath_issuer)
    issuer = re.search(r"(.+)", find_issuer.text).group().encode("utf-8")
    print "Issuer: %s" % issuer
    break
except NoSuchElementException:
    pass

Upvotes: 1

Views: 1409

Answers (2)

alecxe
alecxe

Reputation: 474281

I would make a reusable function to get the value by key:

  • "Exchange Traded Concepts" by "Issuer"
  • "ETF" by "Structure" etc

Example working implementation:

from selenium.common.exceptions import NoSuchElementException

def get_value(driver, key):
    key = key + ":"
    try:
        return driver.find_element_by_xpath("//span[@class='minimal-list__title' and . = '%s']/following-sibling::span" % key).text
    except NoSuchElementException:
        print "Not Found"
        return None

Usage:

get_value(driver, "Issuer")

Upvotes: 1

Madhan
Madhan

Reputation: 5818

You can go with the following xpath

//span[@class='minimal-list__title'][text()='Issuer:']//following-sibling::span[@class='minimal-list__value']

Upvotes: 2

Related Questions