CODEWITHSUNDEEP
pythonseleniumxpathselenium-webdriver
Halcyon Abraham Ramirez
Halcyon Abraham Ramirez

Reputation: 1560

How to select all tags with Selenium and Python

Consider:

<a title="Citrate of Magnesia for Consumers" href="/cdi/citrate-of-magnesia-solution.html">
<b>Citrate of Magnesia</b>

I'm trying to pull data from a medication website, and how do I select all the text in the <b></b> tags?

Because that's the text I want.

I've tried *//a[@b], and it didn't work.

Upvotes: 0

Views: 4032

Answers (1)

alecxe
alecxe

Reputation: 473773

Assuming you are trying to rely on the preceding a element, use following-sibling, example:

//a/following-sibling::b

Python code:

b = driver.find_element_by_xpath("//a/following-sibling::b")
print(b.text)

If you want multiple b tags having a as a preceding element:

for b in driver.find_elements_by_xpath("//a/following-sibling::b"):
    print(b.text)

Solution provided after chatting:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Chrome()
driver.get("http://www.drugs.com/drug-class/laxatives.html?condition_id=&generic=0&sort=rating&order=desc")

# Wait for the table list to load
table = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "table.data-list")))

for b in table.find_elements_by_css_selector("tr td > a[href] > b"):
    print(b.text)

Upvotes: 2

Related Questions