CODEWITHSUNDEEP
pythonseleniumxpathcss-selectorswebdriverwait
xrdty
xrdty

Reputation: 886

Selenium not performing click action

I am trying to scrape a table with multiple pages. The next page is obtained by clicking on the 'Next Page button' (See code snippet). 

<a class="botons" id="btn2" href="javascript:void(0)">
 Next Page  
 <i class="fas fa-long-arrow-alt-right"></i>
</a>

Selenium finds the "button" and has no trouble "clicking" via the following code: 

btn_next = self.browser.find_element_by_partial_link_text("Next Page")
btn_next.click()

However, the page just refreshes and the table doesn't update to its next page.

Any clues to what's going wrong here?

Edit: table can be found at https://www.proxy-list.download/HTTPS

Edit2:

chrome_options = Options()
chrome_options.add_argument("--enable-javascript")
chrome_options.add_argument("--disable-extensions")
chrome_options.add_argument("--disable-gpu")

chrome_options.add_argument("--headless")

Upvotes: 0

Views: 251

Answers (2)

undetected Selenium
undetected Selenium

Reputation: 193348

The desired element is an JavaScript enabled element so to locate and click() on the element you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.botons[id^='btn2'] i.fas.fa-long-arrow-alt-right"))).click()

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='botons' and normalize-space()='Next Page']/i[@class='fas fa-long-arrow-alt-right']"))).click()

  • Note : You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Upvotes: 0

cruisepandey
cruisepandey

Reputation: 29382

There is one id assigned to that button btn2 and it's unique too.

You should give preference to id over link Text.

That said, Next Page link isn't present in view point.For that first you have to move the focus of driver like this : 

wait = WebDriverWait(self.browser,10)
next_page = wait.until(EC.visibility_of_element_located((By.ID, "btn2")))
ActionChains(self.browser).move_to_element(next_page).perform()
next_page.click()

Imports : 

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Upvotes: 1

Related Questions