CODEWITHSUNDEEP
pythonselenium
the star
the star

Reputation: 379

Access the content of the tags with -xpath -selenium -python

I try access to content of some tags in a page.

But they are impossible to access.

Here are the first lines of code:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

driver = webdriver.Chrome()
url = "https://www.flashscore.com/match/hzciHz2I/#h2h;overall"
driver.get(url)

Here is the middle code (the one that pose a problem):

I did two version, here is the first one

data = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH,
"//table[contains(@class,'h2h_mutual')]/tbody/tr[contains(@class,'highlight')]")))

and here is the second one

data = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH,
"//table[@class='h2h_mutual']/tbody/tr[@class='highlight']")))

but it did not find tags and returned an exception 'TimeoutException:'

I not understund why this does not work, someone can help me?

here is the end of the code, that serves to display the content :

for elem in data:
     print(elem.text)

here is first version in full :

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException

driver = webdriver.Chrome()
url = "https://www.flashscore.com/match/hzciHz2I/#h2h;overall"
driver.get(url)

data = WebDriverWait(driver, 10).until(EC.visibility_of_all_elements_located((By.XPATH,
"//table[contains(@class,'h2h_mutual')]/tbody/tr[contains(@class,'highlight')]")))

for elem in data:
     print(elem.text)

I specify that this page is a window open after click on a link in this page and that I did good the switch between the two page and even perform some actions on it before this one. (The firsts lines are so there just for that you can reproduce the error quickly)

this is how I proceeded :

driver.find_element(By.CSS_SELECTOR, "#li2").click()
WebDriverWait(driver, timeout=5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#li2")))
window_first = driver.window_handles[0]
link_page = WebDriverWait(driver, timeout=5).until(EC.element_to_be_clickable((By.XPATH,"//div[text()='Nantes']")))
driver.execute_script("arguments[0].click();", link_page)
Window_second = driver.window_handles[1]
driver.switch_to.window(Window_second)
driver.find_element(By.CSS_SELECTOR, "#a-match-head-2-head").click()
WebDriverWait(driver, timeout=5).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#a-match-head-2-head")))

Upvotes: 1

Views: 1088

Answers (1)

Maximilian Peters
Maximilian Peters

Reputation: 31739

As written in the comment, most of the elements returned by your XPATH expression are not visible, either because the user needs to click on 'show more results' or because they are in a div with visibility: none. If you change your wait to presence_of_all_elements_located, it should work.

data = WebDriverWait(driver, 10).until(EC.presence_of_all_elements_located((By.XPATH,
"//table[contains(@class,'h2h_mutual')]/tbody/tr[contains(@class,'highlight')]")))


print(data[0].text)

>>> '04.02.20 L1 Nantes Paris SG 1 : 2'

Some background: presence_of_all_elements_located check if the elements are located, i.e. existing, on the web page but doesn't need them to be visible. visibility_of_all_elements_located checks for presence and visibility, i.e. one could say it's a more stringent version of presence.

See Selenium Documentation

Upvotes: 1

Related Questions