CODEWITHSUNDEEP
python-3.xselenium
user3798654
user3798654

Reputation: 431

ElementNotVisibleException Python/Selenium error for next page button click

I am trying to figure out how to use selenium to perform a next page click on a news release page. Here is my code that will go to the correct website and perform a search to obtain the correct news release articles topic page. This site is configured so that to see every news release on page 1 after the search has executed you must also select the More News Results button at the bottom of the page. I am able to get to the full page 1 news section without problem. Here is my code that performs the search and page clicks. 

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

browser = webdriver.Chrome()
browser.get('http://www.businesswire.com/portal/site/home/')
search_box_element = browser.find_element_by_id('bw-search-input')
search_box_element.clear()
search_box_element.send_keys('biotechnology')
search_box_element.send_keys(Keys.ENTER)
search_box_element_two = browser.find_element_by_id('more-news-results')
search_box_element_two.click()

That part of the code works fine but I want to be able to click on the next button to move to page 2 and then page 3 and so on. Here is the code that I thought would work but it doesn't:

next_page_click_element = browser.find_element_by_class_name("bw-paging-next")
next_page_click_element.click()

This portion of the code throws an error:

selenium.common.exceptions.ElementNotVisibleException: Message: 
element not visible

I have also tried using 

next_page_click_element = browser.find_element_by_xpath('//*[@id="more-news-pagination"]/div/div[1]/div/a')

but got the same error message. I have also tried using a wait by adding these lines of codes just before the next_page_click_element section.

element_present = EC.presence_of_element_located((By.ID, "bw-paging-next"))
WebDriverWait(browser, 10).until(element_present)

While this does cause the program to wait it returns this error message:

raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

Any suggestions on how to resolve this issue would be greatly appreciated.

Upvotes: 1

Views: 240

Answers (1)

undetected Selenium
undetected Selenium

Reputation: 193248

Here is the Answer to your Question:

A few words about the solution-

  1. As the Page 2,Page 3,Next etc elements are at the bottom of the page, you have to scroll down to bring those elements within the Viewport.
  2. Once you scroll down you may consider to induce some ExplicitWait for the elements to be located properly on the HTML DOM.

  3. Here is your own code with some simple tweaks which will finally scroll to the bottom of the page and click on Next link:

    from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

browser=webdriver.Chrome("C:\\Utility\\BrowserDrivers\\chromedriver.exe")
browser.get('http://www.businesswire.com/portal/site/home/')
search_box_element = browser.find_element_by_id('bw-search-input')
search_box_element.clear()
search_box_element.send_keys('biotechnology')
search_box_element.send_keys(Keys.ENTER)
search_box_element_two = browser.find_element_by_id('more-news-results')
search_box_element_two.click()
last_height = browser.execute_script("return document.body.scrollHeight")
browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
next_page_click_element = WebDriverWait(browser, 10).until(
    EC.presence_of_element_located((By.XPATH, "//div[@id='more-news-pagination']/div/div/div/a[text()='Next']"))
)
next_page_click_element.click()

Let me know if this Answers your Question.

Upvotes: 1

Related Questions