CODEWITHSUNDEEP
pythonseleniumselenium-webdriverfirefoxxpath
Tomasz Tabernacki
Tomasz Tabernacki

Reputation: 15

Selenium Python Find Elements by XPath does not return all expected elements

I am working on a scraping data from a series of tables on an HTML website. The website has varying numbers of tables depending on the input, so I am trying to use: .find_elements_by_xpath("//table") to simply get a list with all the table elements on the page.

However, it is only returning the first table on the page in this case. When I use find_element_by_xpath(//table[2]), it returns the other table, but using find_elements does not.

The website portal is here, just press send at the bottom. (I am trying to get the data from the results page). I am using Selenium in Python on the Firefox Webbrowser.

Interesting note, the header table has an XPath of table[1], which begs the question of where is table[0].

Upvotes: 1

Views: 1777

Answers (1)

rahul rai
rahul rai

Reputation: 2326

The Key here is to wait properly before you navigate to new tab. As after clicking on send a new tab is getting opened. See below code:

driver.get('http://imed.med.ucm.es/Tools/rankpep.html')
WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH, "//img[@src='/icons/ig_logo.png']")))

send = driver.find_element_by_xpath("//input[@value='Send']")
driver.execute_script("arguments[0].scrollIntoView();", send)
driver.execute_script("arguments[0].click();", send)

WebDriverWait(driver, 20).until(EC.number_of_windows_to_be(2)) #Wait for 2nd tab to open
driver.switch_to.window(driver.window_handles[1])
count = driver.find_elements_by_xpath("//table")
print(len(count))

Out Put: As on new page there are two tables, it has printed 2 enter image description here

Upvotes: 2

Related Questions