CODEWITHSUNDEEP
selenium-webdriverxpathwebdriverwebdriverwaitxpath-1.0
janosch1931
janosch1931

Reputation: 13

Webscraping with Selenium using Xpath

I'm looking for some numbers on https://tnnslive.com/

The numbers I'm looking for on tnnslive.com:

the numbers I'm looking for on tnnslive.com

If I copy the xpath in Chrome, I get:

//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]

If I run the following code:

from selenium import webdriver

# Start a new Chrome browser session
driver = webdriver.Chrome(executable_path="PATH_TO_CHROMEDRIVER")

# Navigate to the desired URL
driver.get('https://tnnslive.com/match/VDDGj17PodORnq62YwQk')

# Find the element by its XPath and print its text
element = driver.find_element("xpath", '//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]')    

# Close the browser
driver.quit()

the error occurs:

NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="root"]/div/div/div/div/div/div/div/div[2]/div[2]/div/div/div/div/div[2]/div[2]/div/div[1]/div/div[1]/div/div/div/div/div[3]/div/div/div/div[7]/div/div[2]"}
  (Session info: chrome=115.0.5790.171)

Why? How can I solve this issue?

Upvotes: 1

Views: 56

Answers (1)

undetected Selenium
undetected Selenium

Reputation: 193108

To print the text associated with 1st Serve Points Won for the first player within the website you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:

  • Using XPATH, following-sibling and text attribute:

    driver.get(url='https://tnnslive.com/match/MpVVddDZigie7GHM9ip8')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='1st Serve Points Won']//following-sibling::div[1]"))).text)

  • Using XPATH, following and get_attribute("innerHTML"):

    driver.get(url='https://tnnslive.com/match/MpVVddDZigie7GHM9ip8')
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[text()='1st Serve Points Won']//following::div[1]"))).get_attribute("innerHTML"))

  • Console output:

    64% (29/45)

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

Upvotes: 1

Related Questions