CODEWITHSUNDEEP
pythonselenium-webdriverweb-scraping
Robin_Xu
Robin_Xu

Reputation: 1

How to go to the next page within the same URL of UK Currys?

Hi I want to scrape the data from UK Currys. The web is (https://www.currys.co.uk/appliances/laundry/washing-machines/freestanding-washing-machines).

But different pages have the same URL. I tried to use Selenium or execute_script to turn to the next page. Unfortunately, I could not make it.

I used two method. First one:

next_btn = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='product-search-results']/div[1]/div[2]/div[5]/div[26]/div[1]/div/div[3]/ul/li[6]/a")))
driver.execute_script("arguments[0].click();", next_btn)
driver.quit()

Second one:

next_page = driver.execute_script('return document.querySelector("#product-search-results > div.row.refinement-section > div.col-sm-12.col-lg-9.device-show-width > div.row.product-grid.list-view.justify-content-center > div.col-12.grid-footer > div:nth-child(1) > div > div.pagination-wrapper.pagination.justify-content-center.d-lg-block.d-xl-block.d-none > ul > li.first-last.page-item.pagination-arrrow-next > a")')
driver.execute_script('arguments[0].click();',next_page)

Neither of them could go to the next page.

Upvotes: 0

Views: 96

Answers (1)

Mate Mrše
Mate Mrše

Reputation: 8424

Try first to scroll to button before clicking:

from selenium.webdriver.common.action_chains import ActionChains

next_btn = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[@id='product-search-results']/div[1]/div[2]/div[5]/div[26]/div[1]/div/div[3]/ul/li[6]/a")))
actions = ActionChains(driver)
actions.move_to_element(next_btn).perform()
next_btn.click()

Upvotes: 0

Related Questions