CODEWITHSUNDEEP
htmlseleniumscreen-scraping
Minte
Minte

Reputation: 41

How to find a element in selenium based on text from sibling?

I've got a question concerning scraper selenium.

I've got a bunch of html, but every class has the same name. The only thing that distinguishes one class from the other is the textual value (marked in red). I want to get the element "Show More" for each of these classes. So I thought, to get this element, I'd somehow have to access it through the textual value in the span class _3QYUVo0T.

Anybody has any idea to go about this challenge? I already tried some things with follow-siblings but couldnt quite figure it out. Thanks in advance!

enter image description here

Upvotes: 1

Views: 805

Answers (2)

cruisepandey
cruisepandey

Reputation: 29362

try this to reach to show more from Cuisine :

//span[text()='Cuisine']/../following-sibling::div/descendant::span[text()='Show more']

for Dishes :

 //span[text()='Dishes']/../following-sibling::div/descendant::span[text()='Show more']

Code 1 :

driver.find_element_by_xpath("//span[text()='Cuisine']/../following-sibling::div/descendant::span[text()='Show more']").click()

Code 2 :

With Explicit waits

wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Cuisine']/../following-sibling::div/descendant::span[text()='Show more']"))).click()

Imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

if the above two do not work, try with JS :

button = driver.find_element_by_xpath("//span[text()='Cuisine']/../following-sibling::div/descendant::span[text()='Show more']")
driver.execute_script("arguments[0].click();", button)

Upvotes: 2

Prophet
Prophet

Reputation: 33351

To click Show more element below Dishes you can use this XPath:

//span[text()='Dishes']/../..//span[text()='Show more']

With Selenium find_element in Python it will be:

driver.find_element_by_xpath("//span[text()='Dishes']/../..//span[text()='Show more']").click()

You can use this in more smart and generic way passing the text as parameter.
Also you possibly will have to add a wait before clicking the element to make it loaded.

Upvotes: 1

Related Questions