CODEWITHSUNDEEP
pythonseleniumxpath
Fabien
Fabien

Reputation: 55

Select element using XPATH with Python?

I am trying to determine the number of pages of data generated by the Indian Central Pollution Controal Board. Here is an example of output. Following https://github.com/RachitKamdar/Python-Scraper, I used selenium/python

maxpage = int(browser.find_elements(By.XPATH,"//*[@id='DataTables_Table_0_paginate']/span/a")[-1].text)

but this produces an empty array. I am really not sure what I am doing wrong. Any help would be greatly appreciated. Thanks

Upvotes: 0

Views: 225

Answers (2)

Rohan
Rohan

Reputation: 54

You might want to try getattribute('textContent')

In your case:

maxpage=browser.find_element_by_xpath("(//*[@id='DataTables_Table_0_paginate']/span/a)[last()]").getattribute('textContent')

Upvotes: 0

Prophet
Prophet

Reputation: 33361

You have to add expected condition to wait until the page loaded the data.
You can wait for visibility of element you are using and after that get it's text, like this:

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

wait = WebDriverWait(driver, 20)
wait.until(EC.visibility_of_element_located((By.XPATH, "//*[@id='DataTables_Table_0_paginate']/span/a")))
maxpage = int(browser.find_elements(By.XPATH,"//*[@id='DataTables_Table_0_paginate']/span/a")[-1].text)

Upvotes: 1

Related Questions