CODEWITHSUNDEEP
pythonseleniumscreen-scrapingclickable
Runner Bean
Runner Bean

Reputation: 5165

Python Scraping - Selenium - Element is not clickable at point

I'm trying to click the next button on a webpage multiple times, I need scrape the page after each click. The following code is shortened but illustrates my situation. I am able to scrape the required tag after the first click, but then the second click fails. Here is the code

from selenium import webdriver  
from selenium.common.exceptions import NoSuchElementException  
from selenium.webdriver.common.keys import Keys  

browser = webdriver.Firefox()  
browser.get('http://www.agoda.com/the-coast-resort-koh-phangan/hotel/koh-phangan-th.html') 
browser.find_element_by_xpath("//a[@id='next-page']").click()
browser.find_element_by_xpath("//a[@id='next-page']").click()

browser.quit()

I get this error

WebDriverException: Message: Element is not clickable at point

Although when I use beautifulSoup and get the source code after the first click the element I am trying to click again is clearly there to click.

I've read on other answers I might need to wait some time but not sure how to implement that.

Upvotes: 0

Views: 497

Answers (1)

Paul Collingwood
Paul Collingwood

Reputation: 9116

The first time you click the next page starts to load, you can't click on a link that is on a page that is still loading or on a link on a page that is about to be replaced by another page! So the message is quite correct. So you just have to wait for the next page to appear before you can click on the next link. You can just sleep for N seconds or, much better, wait for the element to appear before clicking on it again.

Have a look at: Expected Conditions

So there you would probably want to use element_to_be_clickable on each attempt to click. What happens is it waits N seconds for it to be clickable and if it does not take that state in that time it throws an exception.

Here is a good blog post on all this: How to get Selenium to wait for page load after a click

Here is a simple example of how you can use that. Here you have just clicked on the link for the first time, and then after that to wait for the next to load you can:

    xpath = "//a[@id='next-page']"
    WebDriverWait(browser.webdriver, 10).until(
        expected_conditions.element_to_be_clickable((By.XPATH, xpath)))
    browser.find_element_by_xpath(xpath).click()

Upvotes: 1

Related Questions