CODEWITHSUNDEEP
pythonseleniumweb-scrapingxpath
tickly potato
tickly potato

Reputation: 122

Python selenium, why my text can't be located by WebDriverWait.until? Text could be located by page_source.find but not find_elements_by_xpath

I'm trying to wait the webpage to fully load before I proceed and find some elements.

1. If I do

EC.presence_of_element_located((By.XPATH, "//*[contains(text(), 'my text 1234567')]"))

I will get

<selenium.webdriver.support.expected_conditions.presence_of_element_located at 0x143304641c0>

This means my text is found right?

2. But If I do

WebDriverWait(driver, 5).until(EC.presence_of_element_located((By.XPATH, "//*[contains(text(), 'my text 1234567')]")))

I will get

selenium.common.exceptions.TimeoutException: Message:

3. Then I checked

driver.find_elements_by_xpath("//*[contains(text(), 'my text 1234567')]")
Out[55]: []

4. If I do

driver.page_source.find('my text 1234567')
Out[64]: 971

I'm very confused. Why these would happen? Should I modify my By.XPATH?

Problem solved following below answer

Changed

"//*[contains(text(), 'my text 1234567')]"

to

"//*[contains(., 'my text 1234567')]"

Why text() here won't work?

Upvotes: 0

Views: 483

Answers (1)

Anna
Anna

Reputation: 124

I'm not sure what your wanted result is. If you simply want to return the text "Securities Exchange Act of 1934", this is an option:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

path = "YOUR PATH HERE\chromedriver.exe"
driver = webdriver.Chrome(path)
wait = WebDriverWait(driver, 5)

driver.get("https://www.sec.gov/Archives/edgar/data/896397/000089639701500011/seh10q2qtr2001.htm")

wait.until(EC.presence_of_element_located((By.TAG_NAME, "p")))
paragraph = driver.find_elements_by_tag_name("p")[1].text
line = paragraph.split("\n")[3]
result = line[60:]
print(result)

Instead, if you want to return the first paragraph where "Securities Exchange Act of 1934" is present, this is an option, following a similar syntax to what you were using:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait

path = "YOUR PATH HERE\chromedriver.exe"
driver = webdriver.Chrome(path)
wait = WebDriverWait(driver, 5)

driver.get("https://www.sec.gov/Archives/edgar/data/896397/000089639701500011/seh10q2qtr2001.htm")

paragraph = wait.until(EC.presence_of_element_located((By.XPATH, "//p[contains(.,'Securities Exchange Act of 1934')]")))
print(paragraph.text)

Upvotes: 1

Related Questions