CODEWITHSUNDEEP
pythonseleniumselenium-webdriverselenium-chromedriver
mark377
mark377

Reputation: 91

Issues with Selenium Click on Link

I'm having a strange issue with Python Selenium when trying to click on the next page link in Curry's PC World Televisions page. See the next button on (https://www.currys.co.uk/tv-and-audio/televisions/tvs)

I am testing in non-headless mode with chrome, so i can see the outline of the next button being highlighted, when actioning the click, but it is not navigating to the next page url as expected.

Here is what i have tried on the page above:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as ec
from selenium.common.exceptions import NoSuchElementException
import time

chrome_options = Options()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=chrome_options)
driver.set_window_size(1920, 1080)
driver.maximize_window()

driver.get('https://www.currys.co.uk/tv-and-audio/televisions/tvs')

next_page_link = WebDriverWait(driver, 10).until(
                 ec.element_to_be_clickable((By.CSS_SELECTOR, 
                                            'ul.pagination '
                                            'li.pagination-arrrow-next a.page-next')))
next_page_link.click()

time.sleep(10)

I have also tried using execute script, (which does not work either):

driver.execute_script('arguments[0].click()', next_page_link)

I am able to extract the href from the link element (so i know its the right element):

href = next_page_link.get_attribute('href')
print("href is: {}".format(href))
# href is: https://www.currys.co.uk/tv-and-audio/televisions/tvs?start=20&sz=20

I'm a bit lost as to what the issue might be here, would appreciate any help!

Upvotes: 1

Views: 72

Answers (1)

Prophet
Prophet

Reputation: 33371

I see the following issues here:

  1. You have to close the accept cookies pop-up.
  2. You need to scroll the page down to the pager elements.
  3. Selenium indeed doesn't make the page to go to the next page BUT even when I click the Next button manually or even any other pagination button there manually the page is not changed. So, the problem is with the web page itself, not with Selenium.
    This is the code I used:
import time

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = 'https://www.currys.co.uk/tv-and-audio/televisions/tvs'
driver.get(url)
wait = WebDriverWait(driver, 20)

wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
next_page_link = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR,'ul.pagination li.pagination-arrrow-next a.page-next')))
next_page_link.location_once_scrolled_into_view
time.sleep(1)
next_page_link.click()

Upvotes: 2

Related Questions